2023-11-27 14:54:03,693 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-11-27 14:54:03,937 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e 2023-11-27 14:54:03,950 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-11-27 14:54:03,951 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/cluster_e76484b8-e34a-cbd1-e5d0-706505e5be75, deleteOnExit=true 2023-11-27 14:54:03,951 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-11-27 14:54:03,951 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/test.cache.data in system properties and HBase conf 2023-11-27 14:54:03,952 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/hadoop.tmp.dir in system properties and HBase conf 2023-11-27 14:54:03,952 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/hadoop.log.dir in system properties and HBase conf 2023-11-27 14:54:03,953 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/mapreduce.cluster.local.dir in system properties and HBase conf 2023-11-27 14:54:03,953 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-11-27 14:54:03,953 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-11-27 14:54:04,081 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-11-27 14:54:04,547 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-11-27 14:54:04,553 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-11-27 14:54:04,554 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-11-27 14:54:04,554 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-11-27 14:54:04,555 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-27 14:54:04,555 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-11-27 14:54:04,556 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-11-27 14:54:04,556 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-27 14:54:04,557 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-27 14:54:04,557 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-11-27 14:54:04,558 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/nfs.dump.dir in system properties and HBase conf 2023-11-27 14:54:04,558 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/java.io.tmpdir in system properties and HBase conf 2023-11-27 14:54:04,558 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-27 14:54:04,559 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-11-27 14:54:04,559 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-11-27 14:54:05,281 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-27 14:54:05,288 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-27 14:54:05,660 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-11-27 14:54:05,876 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-11-27 14:54:05,898 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-27 14:54:05,949 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-11-27 14:54:05,995 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/java.io.tmpdir/Jetty_localhost_45945_hdfs____.jjhxx9/webapp 2023-11-27 14:54:06,148 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45945 2023-11-27 14:54:06,160 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-27 14:54:06,160 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-27 14:54:07,114 WARN [Listener at localhost/46277] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-27 14:54:07,203 WARN [Listener at localhost/46277] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-11-27 14:54:07,229 WARN [Listener at localhost/46277] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-27 14:54:07,237 INFO [Listener at localhost/46277] log.Slf4jLog(67): jetty-6.1.26 2023-11-27 14:54:07,242 INFO [Listener at localhost/46277] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/java.io.tmpdir/Jetty_localhost_43979_datanode____.w271gd/webapp 2023-11-27 14:54:07,345 INFO [Listener at localhost/46277] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43979 2023-11-27 14:54:07,697 WARN [Listener at localhost/37439] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-27 14:54:08,208 DEBUG [Listener at localhost/37439] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e 2023-11-27 14:54:08,215 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x101e2c06222c7b28: Processing first storage report for DS-59361a84-8713-4d44-9277-63fc85eaa867 from datanode 31266597-0116-4265-9ab2-f25c98e274d7 2023-11-27 14:54:08,216 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x101e2c06222c7b28: from storage DS-59361a84-8713-4d44-9277-63fc85eaa867 node DatanodeRegistration(127.0.0.1:45863, datanodeUuid=31266597-0116-4265-9ab2-f25c98e274d7, infoPort=37479, infoSecurePort=0, ipcPort=37439, storageInfo=lv=-57;cid=testClusterID;nsid=1897803726;c=1701096845386), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-11-27 14:54:08,216 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x101e2c06222c7b28: Processing first storage report for DS-8faa821a-c02f-445d-a2e8-55b1d1490b04 from datanode 31266597-0116-4265-9ab2-f25c98e274d7 2023-11-27 14:54:08,217 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x101e2c06222c7b28: from storage DS-8faa821a-c02f-445d-a2e8-55b1d1490b04 node DatanodeRegistration(127.0.0.1:45863, datanodeUuid=31266597-0116-4265-9ab2-f25c98e274d7, infoPort=37479, infoSecurePort=0, ipcPort=37439, storageInfo=lv=-57;cid=testClusterID;nsid=1897803726;c=1701096845386), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-11-27 14:54:08,315 INFO [Listener at localhost/37439] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/cluster_e76484b8-e34a-cbd1-e5d0-706505e5be75/zookeeper_0, clientPort=61098, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/cluster_e76484b8-e34a-cbd1-e5d0-706505e5be75/zookeeper_0/version-2, dataDirSize=457 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/cluster_e76484b8-e34a-cbd1-e5d0-706505e5be75/zookeeper_0/version-2, dataLogSize=457 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, clientPortListenBacklog=-1, serverId=0 2023-11-27 14:54:08,333 INFO [Listener at localhost/37439] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=61098 2023-11-27 14:54:08,340 INFO [Listener at localhost/37439] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-27 14:54:08,343 INFO [Listener at localhost/37439] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-27 14:54:09,022 INFO [Listener at localhost/37439] util.FSUtils(471): Created version file at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb with version=8 2023-11-27 14:54:09,022 INFO [Listener at localhost/37439] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/hbase-staging 2023-11-27 14:54:09,353 INFO [Listener at localhost/37439] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-11-27 14:54:09,802 INFO [Listener at localhost/37439] client.ConnectionUtils(126): master/jenkins-hbase4:0 server-side Connection retries=3 2023-11-27 14:54:09,834 INFO [Listener at localhost/37439] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-27 14:54:09,835 INFO [Listener at localhost/37439] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-27 14:54:09,835 INFO [Listener at localhost/37439] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-27 14:54:09,835 INFO [Listener at localhost/37439] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-27 14:54:09,835 INFO [Listener at localhost/37439] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-27 14:54:09,981 INFO [Listener at localhost/37439] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-11-27 14:54:10,053 DEBUG [Listener at localhost/37439] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-11-27 14:54:10,154 INFO [Listener at localhost/37439] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39835 2023-11-27 14:54:10,163 INFO [Listener at localhost/37439] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-27 14:54:10,165 INFO [Listener at localhost/37439] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-27 14:54:10,188 INFO [Listener at localhost/37439] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39835 connecting to ZooKeeper ensemble=127.0.0.1:61098 2023-11-27 14:54:10,230 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:398350x0, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-27 14:54:10,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39835-0x1002f55dfc60000 connected 2023-11-27 14:54:10,266 DEBUG [Listener at localhost/37439] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-27 14:54:10,267 DEBUG [Listener at localhost/37439] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-27 14:54:10,272 DEBUG [Listener at localhost/37439] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-27 14:54:10,284 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39835 2023-11-27 14:54:10,284 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39835 2023-11-27 14:54:10,285 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39835 2023-11-27 14:54:10,285 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39835 2023-11-27 14:54:10,286 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39835 2023-11-27 14:54:10,291 INFO [Listener at localhost/37439] master.HMaster(444): hbase.rootdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb, hbase.cluster.distributed=false 2023-11-27 14:54:10,361 INFO [Listener at localhost/37439] client.ConnectionUtils(126): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-11-27 14:54:10,362 INFO [Listener at localhost/37439] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-27 14:54:10,362 INFO [Listener at localhost/37439] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-27 14:54:10,362 INFO [Listener at localhost/37439] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-27 14:54:10,362 INFO [Listener at localhost/37439] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-27 14:54:10,362 INFO [Listener at localhost/37439] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-27 14:54:10,367 INFO [Listener at localhost/37439] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-11-27 14:54:10,370 INFO [Listener at localhost/37439] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42669 2023-11-27 14:54:10,372 INFO [Listener at localhost/37439] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-11-27 14:54:10,378 DEBUG [Listener at localhost/37439] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-11-27 14:54:10,379 INFO [Listener at localhost/37439] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-27 14:54:10,382 INFO [Listener at localhost/37439] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-27 14:54:10,384 INFO [Listener at localhost/37439] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42669 connecting to ZooKeeper ensemble=127.0.0.1:61098 2023-11-27 14:54:10,388 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:426690x0, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-27 14:54:10,389 DEBUG [Listener at localhost/37439] zookeeper.ZKUtil(165): regionserver:426690x0, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-27 14:54:10,390 DEBUG [Listener at localhost/37439] zookeeper.ZKUtil(165): regionserver:426690x0, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-27 14:54:10,391 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42669-0x1002f55dfc60001 connected 2023-11-27 14:54:10,392 DEBUG [Listener at localhost/37439] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-27 14:54:10,392 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42669 2023-11-27 14:54:10,392 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42669 2023-11-27 14:54:10,393 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42669 2023-11-27 14:54:10,396 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42669 2023-11-27 14:54:10,396 DEBUG [Listener at localhost/37439] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42669 2023-11-27 14:54:10,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:54:10,408 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-27 14:54:10,410 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:54:10,434 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-27 14:54:10,435 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-27 14:54:10,435 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:54:10,436 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-27 14:54:10,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,39835,1701096849177 from backup master directory 2023-11-27 14:54:10,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-27 14:54:10,441 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:54:10,441 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-27 14:54:10,442 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-27 14:54:10,442 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:54:10,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-11-27 14:54:10,446 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-11-27 14:54:10,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/hbase.id with ID: e5712791-1270-4aa0-9821-0ee6b96a4c43 2023-11-27 14:54:10,565 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-27 14:54:10,580 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:54:10,617 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x55df12ba to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:54:10,655 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@33bf3d21, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:54:10,680 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-11-27 14:54:10,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-11-27 14:54:10,702 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-11-27 14:54:10,703 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-11-27 14:54:10,705 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-27 14:54:10,708 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-27 14:54:10,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-27 14:54:10,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/data/master/store-tmp 2023-11-27 14:54:10,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:10,777 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-27 14:54:10,777 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-27 14:54:10,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-27 14:54:10,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-27 14:54:10,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-27 14:54:10,778 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-27 14:54:10,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-27 14:54:10,779 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/WALs/jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:54:10,799 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39835%2C1701096849177, suffix=, logDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/WALs/jenkins-hbase4.apache.org,39835,1701096849177, archiveDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/oldWALs, maxLogs=10 2023-11-27 14:54:10,859 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45863,DS-59361a84-8713-4d44-9277-63fc85eaa867,DISK] 2023-11-27 14:54:10,866 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-11-27 14:54:10,924 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/WALs/jenkins-hbase4.apache.org,39835,1701096849177/jenkins-hbase4.apache.org%2C39835%2C1701096849177.1701096850809 2023-11-27 14:54:10,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45863,DS-59361a84-8713-4d44-9277-63fc85eaa867,DISK]] 2023-11-27 14:54:10,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:54:10,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:10,929 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-11-27 14:54:10,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-11-27 14:54:10,985 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-11-27 14:54:10,991 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-11-27 14:54:11,014 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-11-27 14:54:11,026 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:11,032 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-27 14:54:11,034 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-27 14:54:11,052 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-11-27 14:54:11,056 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:11,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72376589, jitterRate=0.07849521934986115}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:54:11,058 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-27 14:54:11,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-11-27 14:54:11,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-11-27 14:54:11,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-11-27 14:54:11,080 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-27 14:54:11,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 1 msec 2023-11-27 14:54:11,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 32 msec 2023-11-27 14:54:11,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-11-27 14:54:11,140 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-11-27 14:54:11,145 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-11-27 14:54:11,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-11-27 14:54:11,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-11-27 14:54:11,175 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-11-27 14:54:11,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-11-27 14:54:11,185 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-11-27 14:54:11,189 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:54:11,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-11-27 14:54:11,191 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-11-27 14:54:11,206 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-11-27 14:54:11,213 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-27 14:54:11,213 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-27 14:54:11,214 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:54:11,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39835,1701096849177, sessionid=0x1002f55dfc60000, setting cluster-up flag (Was=false) 2023-11-27 14:54:11,238 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:54:11,247 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-11-27 14:54:11,249 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:54:11,255 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:54:11,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-11-27 14:54:11,263 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:54:11,265 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(304): Couldn't delete working snapshot directory: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp 2023-11-27 14:54:11,301 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(954): ClusterId : e5712791-1270-4aa0-9821-0ee6b96a4c43 2023-11-27 14:54:11,307 DEBUG [RS:0;jenkins-hbase4:42669] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-11-27 14:54:11,315 DEBUG [RS:0;jenkins-hbase4:42669] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-11-27 14:54:11,315 DEBUG [RS:0;jenkins-hbase4:42669] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-11-27 14:54:11,321 DEBUG [RS:0;jenkins-hbase4:42669] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-11-27 14:54:11,323 DEBUG [RS:0;jenkins-hbase4:42669] zookeeper.ReadOnlyZKClient(139): Connect 0x4085df3f to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:54:11,328 DEBUG [RS:0;jenkins-hbase4:42669] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e42295f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:54:11,329 DEBUG [RS:0;jenkins-hbase4:42669] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4a7db44f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-27 14:54:11,351 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42669 2023-11-27 14:54:11,355 INFO [RS:0;jenkins-hbase4:42669] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-11-27 14:54:11,355 INFO [RS:0;jenkins-hbase4:42669] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-11-27 14:54:11,355 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1025): About to register with Master. 2023-11-27 14:54:11,358 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(2814): reportForDuty to master=jenkins-hbase4.apache.org,39835,1701096849177 with isa=jenkins-hbase4.apache.org/172.31.14.131:42669, startcode=1701096850360 2023-11-27 14:54:11,373 DEBUG [RS:0;jenkins-hbase4:42669] ipc.RpcConnection(122): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-11-27 14:54:11,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-11-27 14:54:11,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-27 14:54:11,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-27 14:54:11,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-27 14:54:11,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-27 14:54:11,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-11-27 14:54:11,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-27 14:54:11,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1701096881401 2023-11-27 14:54:11,403 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-11-27 14:54:11,407 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-11-27 14:54:11,408 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-11-27 14:54:11,414 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-27 14:54:11,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-11-27 14:54:11,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-11-27 14:54:11,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-11-27 14:54:11,427 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-11-27 14:54:11,427 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-11-27 14:54:11,428 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44349, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-11-27 14:54:11,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,431 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-11-27 14:54:11,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-11-27 14:54:11,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-11-27 14:54:11,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-11-27 14:54:11,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-11-27 14:54:11,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1701096851438,5,FailOnTimeoutGroup] 2023-11-27 14:54:11,438 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39835] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-11-27 14:54:11,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1701096851438,5,FailOnTimeoutGroup] 2023-11-27 14:54:11,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-11-27 14:54:11,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,451 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-27 14:54:11,452 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-27 14:54:11,452 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb 2023-11-27 14:54:11,465 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:11,468 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-27 14:54:11,471 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/info 2023-11-27 14:54:11,472 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-27 14:54:11,473 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:11,473 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-27 14:54:11,475 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(2833): Master is not running yet 2023-11-27 14:54:11,476 WARN [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1033): reportForDuty failed; sleeping 100 ms and then retrying. 2023-11-27 14:54:11,477 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/rep_barrier 2023-11-27 14:54:11,477 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-27 14:54:11,478 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:11,479 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-27 14:54:11,481 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/table 2023-11-27 14:54:11,481 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-27 14:54:11,482 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:11,484 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740 2023-11-27 14:54:11,485 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740 2023-11-27 14:54:11,489 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-27 14:54:11,491 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-27 14:54:11,495 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:11,496 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70258416, jitterRate=0.04693198204040527}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-27 14:54:11,496 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-27 14:54:11,496 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-27 14:54:11,496 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-27 14:54:11,496 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-27 14:54:11,496 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-27 14:54:11,496 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-27 14:54:11,497 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-27 14:54:11,497 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-27 14:54:11,501 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-11-27 14:54:11,501 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-11-27 14:54:11,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-11-27 14:54:11,522 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-11-27 14:54:11,525 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-11-27 14:54:11,577 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(2814): reportForDuty to master=jenkins-hbase4.apache.org,39835,1701096849177 with isa=jenkins-hbase4.apache.org/172.31.14.131:42669, startcode=1701096850360 2023-11-27 14:54:11,583 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39835] master.ServerManager(388): Registering regionserver=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:11,589 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1598): Config from master: hbase.rootdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb 2023-11-27 14:54:11,589 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1598): Config from master: fs.defaultFS=hdfs://localhost:46277 2023-11-27 14:54:11,589 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1598): Config from master: hbase.master.info.port=-1 2023-11-27 14:54:11,594 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-27 14:54:11,595 DEBUG [RS:0;jenkins-hbase4:42669] zookeeper.ZKUtil(163): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:11,595 WARN [RS:0;jenkins-hbase4:42669] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-27 14:54:11,596 INFO [RS:0;jenkins-hbase4:42669] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-27 14:54:11,596 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1951): logDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/WALs/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:11,597 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42669,1701096850360] 2023-11-27 14:54:11,605 DEBUG [RS:0;jenkins-hbase4:42669] zookeeper.ZKUtil(163): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:11,615 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-11-27 14:54:11,623 INFO [RS:0;jenkins-hbase4:42669] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-11-27 14:54:11,640 INFO [RS:0;jenkins-hbase4:42669] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-11-27 14:54:11,644 INFO [RS:0;jenkins-hbase4:42669] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:54:11,644 INFO [RS:0;jenkins-hbase4:42669] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,645 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer$CompactionChecker(1840): CompactionChecker runs every PT1S 2023-11-27 14:54:11,651 INFO [RS:0;jenkins-hbase4:42669] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,651 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,652 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,652 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,652 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,652 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,652 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-27 14:54:11,652 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,652 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,653 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,653 DEBUG [RS:0;jenkins-hbase4:42669] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-27 14:54:11,653 INFO [RS:0;jenkins-hbase4:42669] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,654 INFO [RS:0;jenkins-hbase4:42669] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,654 INFO [RS:0;jenkins-hbase4:42669] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,669 INFO [RS:0;jenkins-hbase4:42669] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-11-27 14:54:11,671 INFO [RS:0;jenkins-hbase4:42669] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42669,1701096850360-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:11,676 DEBUG [jenkins-hbase4:39835] assignment.AssignmentManager(2186): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-11-27 14:54:11,678 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42669,1701096850360, state=OPENING 2023-11-27 14:54:11,686 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-11-27 14:54:11,687 INFO [RS:0;jenkins-hbase4:42669] regionserver.Replication(203): jenkins-hbase4.apache.org,42669,1701096850360 started 2023-11-27 14:54:11,687 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1640): Serving as jenkins-hbase4.apache.org,42669,1701096850360, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42669, sessionid=0x1002f55dfc60001 2023-11-27 14:54:11,687 DEBUG [RS:0;jenkins-hbase4:42669] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-11-27 14:54:11,687 DEBUG [RS:0;jenkins-hbase4:42669] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:11,687 DEBUG [RS:0;jenkins-hbase4:42669] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42669,1701096850360' 2023-11-27 14:54:11,687 DEBUG [RS:0;jenkins-hbase4:42669] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:11,688 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:54:11,688 DEBUG [RS:0;jenkins-hbase4:42669] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:11,689 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-27 14:54:11,689 DEBUG [RS:0;jenkins-hbase4:42669] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-11-27 14:54:11,689 DEBUG [RS:0;jenkins-hbase4:42669] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-11-27 14:54:11,689 DEBUG [RS:0;jenkins-hbase4:42669] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:11,689 DEBUG [RS:0;jenkins-hbase4:42669] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42669,1701096850360' 2023-11-27 14:54:11,689 DEBUG [RS:0;jenkins-hbase4:42669] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-27 14:54:11,690 DEBUG [RS:0;jenkins-hbase4:42669] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-27 14:54:11,690 DEBUG [RS:0;jenkins-hbase4:42669] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-11-27 14:54:11,690 INFO [RS:0;jenkins-hbase4:42669] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-11-27 14:54:11,691 INFO [RS:0;jenkins-hbase4:42669] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-11-27 14:54:11,692 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:11,802 INFO [RS:0;jenkins-hbase4:42669] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42669%2C1701096850360, suffix=, logDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/WALs/jenkins-hbase4.apache.org,42669,1701096850360, archiveDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/oldWALs, maxLogs=32 2023-11-27 14:54:11,820 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45863,DS-59361a84-8713-4d44-9277-63fc85eaa867,DISK] 2023-11-27 14:54:11,823 INFO [RS:0;jenkins-hbase4:42669] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/WALs/jenkins-hbase4.apache.org,42669,1701096850360/jenkins-hbase4.apache.org%2C42669%2C1701096850360.1701096851804 2023-11-27 14:54:11,824 DEBUG [RS:0;jenkins-hbase4:42669] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45863,DS-59361a84-8713-4d44-9277-63fc85eaa867,DISK]] 2023-11-27 14:54:11,874 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:11,876 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-27 14:54:11,879 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59836, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-27 14:54:11,891 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-11-27 14:54:11,891 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-27 14:54:11,894 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42669%2C1701096850360.meta, suffix=.meta, logDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/WALs/jenkins-hbase4.apache.org,42669,1701096850360, archiveDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/oldWALs, maxLogs=32 2023-11-27 14:54:11,910 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45863,DS-59361a84-8713-4d44-9277-63fc85eaa867,DISK] 2023-11-27 14:54:11,914 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/WALs/jenkins-hbase4.apache.org,42669,1701096850360/jenkins-hbase4.apache.org%2C42669%2C1701096850360.meta.1701096851896.meta 2023-11-27 14:54:11,918 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45863,DS-59361a84-8713-4d44-9277-63fc85eaa867,DISK]] 2023-11-27 14:54:11,918 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:54:11,920 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-11-27 14:54:11,936 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-11-27 14:54:11,942 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-11-27 14:54:11,948 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-11-27 14:54:11,948 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:11,948 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-11-27 14:54:11,948 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-11-27 14:54:11,951 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-27 14:54:11,953 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/info 2023-11-27 14:54:11,953 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/info 2023-11-27 14:54:11,954 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-27 14:54:11,955 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:11,955 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-27 14:54:11,956 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/rep_barrier 2023-11-27 14:54:11,957 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/rep_barrier 2023-11-27 14:54:11,957 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-27 14:54:11,958 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:11,958 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-27 14:54:11,960 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/table 2023-11-27 14:54:11,960 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/table 2023-11-27 14:54:11,960 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-27 14:54:11,961 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:11,963 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740 2023-11-27 14:54:11,965 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740 2023-11-27 14:54:11,969 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-27 14:54:11,972 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-27 14:54:11,973 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61290531, jitterRate=-0.08669991791248322}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-27 14:54:11,974 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-27 14:54:11,985 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1701096851866 2023-11-27 14:54:12,001 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for hbase:meta,,1.1588230740 2023-11-27 14:54:12,002 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-11-27 14:54:12,003 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42669,1701096850360, state=OPEN 2023-11-27 14:54:12,011 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-11-27 14:54:12,011 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-27 14:54:12,015 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=3, resume processing ppid=2 2023-11-27 14:54:12,015 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42669,1701096850360 in 319 msec 2023-11-27 14:54:12,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=2, resume processing ppid=1 2023-11-27 14:54:12,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 506 msec 2023-11-27 14:54:12,025 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 713 msec 2023-11-27 14:54:12,025 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1701096852025, completionTime=-1 2023-11-27 14:54:12,026 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(808): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-11-27 14:54:12,026 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1527): Joining cluster... 2023-11-27 14:54:12,088 DEBUG [hconnection-0x697684f-shared-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:54:12,092 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59852, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:54:12,108 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1539): Number of RegionServers=1 2023-11-27 14:54:12,108 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1701096912108 2023-11-27 14:54:12,108 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1701096972108 2023-11-27 14:54:12,108 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1546): Joined the cluster in 81 msec 2023-11-27 14:54:12,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39835,1701096849177-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:12,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39835,1701096849177-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:12,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39835,1701096849177-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:12,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39835, period=300000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:12,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-11-27 14:54:12,138 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-11-27 14:54:12,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-11-27 14:54:12,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-27 14:54:12,157 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-11-27 14:54:12,159 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:54:12,162 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:54:12,185 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:54:12,187 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f empty. 2023-11-27 14:54:12,188 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:54:12,188 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-11-27 14:54:12,611 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-11-27 14:54:12,613 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 686970f6b8c01e84ad20c72c7cb3698f, NAME => 'hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:54:12,624 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:12,624 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 686970f6b8c01e84ad20c72c7cb3698f, disabling compactions & flushes 2023-11-27 14:54:12,624 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:54:12,624 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:54:12,624 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. after waiting 0 ms 2023-11-27 14:54:12,624 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:54:12,624 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:54:12,624 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 686970f6b8c01e84ad20c72c7cb3698f: 2023-11-27 14:54:12,629 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:54:12,643 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1701096852632"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096852632"}]},"ts":"1701096852632"} 2023-11-27 14:54:12,669 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-27 14:54:12,671 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:54:12,675 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096852671"}]},"ts":"1701096852671"} 2023-11-27 14:54:12,679 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-11-27 14:54:12,688 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=686970f6b8c01e84ad20c72c7cb3698f, ASSIGN}] 2023-11-27 14:54:12,690 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=686970f6b8c01e84ad20c72c7cb3698f, ASSIGN 2023-11-27 14:54:12,692 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=686970f6b8c01e84ad20c72c7cb3698f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:54:12,843 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=686970f6b8c01e84ad20c72c7cb3698f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:12,843 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1701096852843"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096852843"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096852843"}]},"ts":"1701096852843"} 2023-11-27 14:54:12,850 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 686970f6b8c01e84ad20c72c7cb3698f, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:13,004 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:13,011 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:54:13,011 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 686970f6b8c01e84ad20c72c7cb3698f, NAME => 'hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:54:13,012 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:54:13,012 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:13,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:54:13,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:54:13,015 INFO [StoreOpener-686970f6b8c01e84ad20c72c7cb3698f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:54:13,017 DEBUG [StoreOpener-686970f6b8c01e84ad20c72c7cb3698f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f/info 2023-11-27 14:54:13,017 DEBUG [StoreOpener-686970f6b8c01e84ad20c72c7cb3698f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f/info 2023-11-27 14:54:13,017 INFO [StoreOpener-686970f6b8c01e84ad20c72c7cb3698f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 686970f6b8c01e84ad20c72c7cb3698f columnFamilyName info 2023-11-27 14:54:13,018 INFO [StoreOpener-686970f6b8c01e84ad20c72c7cb3698f-1] regionserver.HStore(310): Store=686970f6b8c01e84ad20c72c7cb3698f/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:13,020 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:54:13,021 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:54:13,026 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:54:13,029 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:13,030 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 686970f6b8c01e84ad20c72c7cb3698f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61033680, jitterRate=-0.09052729606628418}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:54:13,031 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 686970f6b8c01e84ad20c72c7cb3698f: 2023-11-27 14:54:13,033 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f., pid=6, masterSystemTime=1701096853004 2023-11-27 14:54:13,037 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:54:13,037 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:54:13,038 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=686970f6b8c01e84ad20c72c7cb3698f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:13,039 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1701096853038"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096853038"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096853038"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096853038"}]},"ts":"1701096853038"} 2023-11-27 14:54:13,046 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=6, resume processing ppid=5 2023-11-27 14:54:13,047 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 686970f6b8c01e84ad20c72c7cb3698f, server=jenkins-hbase4.apache.org,42669,1701096850360 in 192 msec 2023-11-27 14:54:13,050 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=5, resume processing ppid=4 2023-11-27 14:54:13,050 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=686970f6b8c01e84ad20c72c7cb3698f, ASSIGN in 358 msec 2023-11-27 14:54:13,051 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:54:13,052 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096853052"}]},"ts":"1701096853052"} 2023-11-27 14:54:13,055 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-11-27 14:54:13,059 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:54:13,060 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-11-27 14:54:13,061 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-11-27 14:54:13,062 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:54:13,062 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 910 msec 2023-11-27 14:54:13,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-11-27 14:54:13,120 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-27 14:54:13,127 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 38 msec 2023-11-27 14:54:13,135 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-11-27 14:54:13,148 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-27 14:54:13,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-11-27 14:54:13,162 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-11-27 14:54:13,164 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-11-27 14:54:13,165 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.723sec 2023-11-27 14:54:13,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-11-27 14:54:13,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-11-27 14:54:13,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-11-27 14:54:13,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39835,1701096849177-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-11-27 14:54:13,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39835,1701096849177-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-11-27 14:54:13,180 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-11-27 14:54:13,209 DEBUG [Listener at localhost/37439] zookeeper.ReadOnlyZKClient(139): Connect 0x36d9c5a6 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:54:13,215 DEBUG [Listener at localhost/37439] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35f0769d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:54:13,231 DEBUG [hconnection-0x5c51df4d-shared-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:54:13,247 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35216, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:54:13,257 INFO [Listener at localhost/37439] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:54:13,258 DEBUG [Listener at localhost/37439] zookeeper.ReadOnlyZKClient(139): Connect 0x65c25ebc to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:54:13,264 DEBUG [Listener at localhost/37439] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@517aa6c4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:54:13,264 INFO [Listener at localhost/37439] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-27 14:54:13,316 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=215, OpenFileDescriptor=505, MaxFileDescriptor=60000, SystemLoadAverage=337, ProcessCount=170, AvailableMemoryMB=8450 2023-11-27 14:54:13,327 DEBUG [Listener at localhost/37439] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:54:13,331 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58462, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:54:13,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-27 14:54:13,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:13,350 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:54:13,352 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:54:13,355 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:13,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-11-27 14:54:13,356 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0 empty. 2023-11-27 14:54:13,358 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:13,358 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-11-27 14:54:13,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-27 14:54:13,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-27 14:54:13,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-27 14:54:13,779 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-11-27 14:54:13,782 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 73f02d3ec84561cf82fbd52d7e1039a0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:54:13,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-27 14:54:14,196 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:14,196 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 73f02d3ec84561cf82fbd52d7e1039a0, disabling compactions & flushes 2023-11-27 14:54:14,196 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:14,196 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:14,196 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. after waiting 0 ms 2023-11-27 14:54:14,196 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:14,196 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:14,197 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:14,201 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:54:14,203 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701096854202"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096854202"}]},"ts":"1701096854202"} 2023-11-27 14:54:14,206 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-27 14:54:14,207 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:54:14,207 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096854207"}]},"ts":"1701096854207"} 2023-11-27 14:54:14,210 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-11-27 14:54:14,214 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=73f02d3ec84561cf82fbd52d7e1039a0, ASSIGN}] 2023-11-27 14:54:14,216 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=73f02d3ec84561cf82fbd52d7e1039a0, ASSIGN 2023-11-27 14:54:14,218 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=73f02d3ec84561cf82fbd52d7e1039a0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:54:14,369 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=73f02d3ec84561cf82fbd52d7e1039a0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:14,370 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701096854369"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096854369"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096854369"}]},"ts":"1701096854369"} 2023-11-27 14:54:14,373 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 73f02d3ec84561cf82fbd52d7e1039a0, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:14,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-27 14:54:14,526 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:14,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:14,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 73f02d3ec84561cf82fbd52d7e1039a0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:54:14,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:14,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:14,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:14,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:14,537 INFO [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:14,539 DEBUG [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:14,540 DEBUG [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:14,544 INFO [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73f02d3ec84561cf82fbd52d7e1039a0 columnFamilyName family1 2023-11-27 14:54:14,547 INFO [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] regionserver.HStore(310): Store=73f02d3ec84561cf82fbd52d7e1039a0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:14,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:14,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:14,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:14,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:14,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 73f02d3ec84561cf82fbd52d7e1039a0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59716037, jitterRate=-0.11016170680522919}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:54:14,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:14,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0., pid=11, masterSystemTime=1701096854526 2023-11-27 14:54:14,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:14,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:14,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=73f02d3ec84561cf82fbd52d7e1039a0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:14,569 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701096854568"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096854568"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096854568"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096854568"}]},"ts":"1701096854568"} 2023-11-27 14:54:14,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=11, resume processing ppid=10 2023-11-27 14:54:14,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 73f02d3ec84561cf82fbd52d7e1039a0, server=jenkins-hbase4.apache.org,42669,1701096850360 in 199 msec 2023-11-27 14:54:14,580 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=10, resume processing ppid=9 2023-11-27 14:54:14,580 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=73f02d3ec84561cf82fbd52d7e1039a0, ASSIGN in 362 msec 2023-11-27 14:54:14,581 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:54:14,582 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096854581"}]},"ts":"1701096854581"} 2023-11-27 14:54:14,584 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-11-27 14:54:14,592 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:54:14,594 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.2470 sec 2023-11-27 14:54:15,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-27 14:54:15,484 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-11-27 14:54:15,501 DEBUG [hconnection-0x5339b42f-metaLookup-shared--pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:54:15,504 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35228, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:54:15,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:15,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:15,648 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:15,648 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:15,649 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:15,649 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:15,650 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:15,650 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:15,651 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:15,651 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:15,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:15,652 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:15,652 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:15,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:15,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:15,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:15,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:15,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:15,655 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:15,656 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:15,656 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:15,657 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:15,657 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:15,658 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:15,658 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:15,658 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:15,659 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-27 14:54:16,218 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130bee1a9b42391452db2a7e5481c0cd989 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130bee1a9b42391452db2a7e5481c0cd989 2023-11-27 14:54:16,236 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/c75b799ab0ba44ef935eae0b27030c9b 2023-11-27 14:54:16,247 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/c75b799ab0ba44ef935eae0b27030c9b as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/c75b799ab0ba44ef935eae0b27030c9b 2023-11-27 14:54:16,256 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/c75b799ab0ba44ef935eae0b27030c9b, entries=1000, sequenceid=5, filesize=175.8 K 2023-11-27 14:54:16,259 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 600ms, sequenceid=5, compaction requested=false 2023-11-27 14:54:16,261 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:16,264 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:16,264 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:16,264 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:16,264 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:16,264 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,264 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:16,264 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:16,268 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,268 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:16,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:16,268 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,268 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:16,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:16,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:16,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:16,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:16,271 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:16,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12b7f0f1[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:16,271 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,273 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,273 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:16,273 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:16,273 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:16,274 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,274 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:16,277 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:16,277 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,277 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:16,278 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:16,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:16,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:16,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:16,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:16,282 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:16,282 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:16,282 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:16,282 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,282 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:16,285 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,285 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:16,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:16,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,285 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:16,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:16,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:16,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:16,286 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:16,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,288 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:16,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,297 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,297 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,297 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:16,297 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:16,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:16,298 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:16,298 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,298 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,298 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,298 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,298 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:16,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:16,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:16,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:16,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:16,300 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:16,300 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:16,401 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:16,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:16,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:16,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:16,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,420 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:16,420 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:16,420 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:16,420 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:16,421 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,421 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,426 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:16,426 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:16,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:16,426 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:16,426 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:16,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:16,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,428 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,428 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-27 14:54:16,428 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:16,428 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:16,428 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:16,429 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:16,429 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:16,429 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:16,429 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:16,429 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:16,429 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-27 14:54:16,858 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511285e4c283308754ac1b63fdb300a940e63 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285e4c283308754ac1b63fdb300a940e63 2023-11-27 14:54:17,272 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/8fbbe081374e45338de1e16bac00d2cd 2023-11-27 14:54:17,280 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/8fbbe081374e45338de1e16bac00d2cd as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/8fbbe081374e45338de1e16bac00d2cd 2023-11-27 14:54:17,288 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/8fbbe081374e45338de1e16bac00d2cd, entries=1, sequenceid=9, filesize=5.0 K 2023-11-27 14:54:17,289 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 860ms, sequenceid=9, compaction requested=false 2023-11-27 14:54:17,289 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:17,289 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:17,290 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:17,290 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:17,290 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,290 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:17,290 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:17,293 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,293 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:17,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:17,293 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,293 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:17,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:17,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:17,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:17,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:17,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5519ce97[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:17,296 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:17,296 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,297 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,297 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:17,298 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:17,298 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,298 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:17,300 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:17,300 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,300 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:17,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:17,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:17,300 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:17,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:17,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:17,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:17,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:17,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:17,304 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:17,304 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:17,304 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:17,304 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,304 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:17,307 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,307 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,307 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:17,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:17,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:17,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:17,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:17,308 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,310 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:17,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,315 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,315 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:17,315 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:17,315 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:17,315 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:17,315 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:17,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:17,315 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:17,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:17,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:17,316 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:17,316 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:17,416 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:17,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:17,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:17,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:17,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,430 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:17,430 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:17,430 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:17,430 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:17,431 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,431 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,432 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,433 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:17,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:17,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:17,433 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:17,433 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:17,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:17,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,434 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-27 14:54:17,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:17,435 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:17,435 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:17,435 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:17,435 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:17,435 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:17,435 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:17,435 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:17,436 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:17,458 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:54:17,620 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-11-27 14:54:17,621 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-11-27 14:54:17,865 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512019d040883cbac41bda6ddfd047e26b631 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512019d040883cbac41bda6ddfd047e26b631 2023-11-27 14:54:18,281 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/aee7cafffd7f42c0937d4cbf2e2d5b6f 2023-11-27 14:54:18,291 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/aee7cafffd7f42c0937d4cbf2e2d5b6f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/aee7cafffd7f42c0937d4cbf2e2d5b6f 2023-11-27 14:54:18,300 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/aee7cafffd7f42c0937d4cbf2e2d5b6f, entries=1, sequenceid=13, filesize=5.0 K 2023-11-27 14:54:18,301 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 865ms, sequenceid=13, compaction requested=true 2023-11-27 14:54:18,301 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:18,301 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:18,301 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:18,302 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:18,302 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,302 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:18,302 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:18,304 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,304 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:18,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:18,305 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:18,305 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:18,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:18,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:18,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:18,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4cd638f7[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:18,307 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:18,308 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,310 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:18,311 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,311 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:18,311 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:18,311 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:18,313 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:18,313 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,313 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:18,313 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:18,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:18,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:18,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:18,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:18,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:18,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:18,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:18,318 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:18,318 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:18,318 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:18,318 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,318 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:18,321 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,321 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:18,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,321 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:18,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:18,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:18,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:18,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:18,323 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:18,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,326 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:18,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,331 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:18,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:18,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:18,332 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,332 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:18,332 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,332 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:18,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:18,332 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,333 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,333 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-27 14:54:18,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:18,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:18,333 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:18,333 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,334 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:18,334 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:18,434 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:18,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:18,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:18,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:18,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,445 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:18,445 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:18,446 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:18,446 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:18,447 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,447 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,449 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:18,449 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:18,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:18,450 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,450 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:18,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,451 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:18,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,451 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,451 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-27 14:54:18,452 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,452 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:18,452 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:18,452 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:18,452 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:18,452 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:18,452 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:18,453 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:18,453 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:18,878 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f1737397861e4820a07cfc0c4bca858d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205f1737397861e4820a07cfc0c4bca858d 2023-11-27 14:54:18,895 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/f853f0671298496fa14517c21650fef9 2023-11-27 14:54:18,904 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/f853f0671298496fa14517c21650fef9 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f853f0671298496fa14517c21650fef9 2023-11-27 14:54:18,913 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f853f0671298496fa14517c21650fef9, entries=1, sequenceid=17, filesize=5.0 K 2023-11-27 14:54:18,914 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 461ms, sequenceid=17, compaction requested=true 2023-11-27 14:54:18,914 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:18,914 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:18,914 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:18,916 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:18,916 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,916 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:18,916 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:18,919 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,919 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:18,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:18,920 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,920 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:18,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:18,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:18,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:18,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:18,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f553b48[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:18,922 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:18,923 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,924 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:18,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:18,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:18,924 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,924 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:18,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:18,926 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:18,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:18,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:18,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:18,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:18,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:18,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:18,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:18,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:18,931 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:18,931 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:18,931 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:18,931 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,931 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:18,936 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,936 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:18,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:18,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,936 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:18,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:18,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:18,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:18,937 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:18,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,939 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:18,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:18,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:18,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:18,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:18,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:18,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:18,944 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:18,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:18,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:18,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:18,945 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,945 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:18,945 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:18,945 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:19,045 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:19,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:19,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:19,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:19,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,057 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:19,057 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:19,058 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:19,058 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:19,058 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,059 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,060 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,060 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:19,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:19,061 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,061 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:19,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:19,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,062 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-27 14:54:19,062 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,062 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:19,062 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:19,063 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:19,063 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:19,063 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:19,063 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:19,349 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-27 14:54:19,350 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-11-27 14:54:19,487 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228f5a8e17fea4a4f0ba2c33c0c3efaae15 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228f5a8e17fea4a4f0ba2c33c0c3efaae15 2023-11-27 14:54:19,500 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/d075b26184d040b69849713958106385 2023-11-27 14:54:19,508 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/d075b26184d040b69849713958106385 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/d075b26184d040b69849713958106385 2023-11-27 14:54:19,515 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/d075b26184d040b69849713958106385, entries=1, sequenceid=21, filesize=5.0 K 2023-11-27 14:54:19,516 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 453ms, sequenceid=21, compaction requested=true 2023-11-27 14:54:19,516 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:19,516 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:19,516 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:19,516 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:19,516 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,517 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:19,517 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:19,520 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,520 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:19,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:19,521 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,521 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:19,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:19,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:19,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:19,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:19,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42486e2b[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:19,523 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:19,523 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,524 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,524 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:19,525 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:19,525 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:19,525 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,525 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:19,526 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,526 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:19,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:19,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:19,526 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:19,527 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:19,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:19,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:19,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:19,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:19,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:19,530 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:19,530 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:19,530 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:19,531 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,531 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:19,532 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,532 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,532 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:19,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:19,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:19,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:19,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:19,533 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:19,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:19,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,535 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:19,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:19,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:19,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:19,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:19,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:19,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-27 14:54:19,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:19,539 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:19,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:19,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:19,540 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,540 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:19,541 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:19,641 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:19,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:19,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:19,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:19,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,654 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:19,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:19,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:19,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:19,655 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,655 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,657 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,657 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:19,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:19,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:19,657 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:19,657 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:19,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,658 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:19,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,658 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,658 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-27 14:54:19,658 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:19,659 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:19,659 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:19,659 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:19,659 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:19,659 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:19,659 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:19,659 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:19,660 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:20,083 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311f8c4587931b44f89e2df6ebac82168a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512311f8c4587931b44f89e2df6ebac82168a 2023-11-27 14:54:20,495 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/1d735ed356c840d1b833f45f822bf41b 2023-11-27 14:54:20,505 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/1d735ed356c840d1b833f45f822bf41b as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/1d735ed356c840d1b833f45f822bf41b 2023-11-27 14:54:20,514 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/1d735ed356c840d1b833f45f822bf41b, entries=1, sequenceid=25, filesize=5.0 K 2023-11-27 14:54:20,515 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 855ms, sequenceid=25, compaction requested=true 2023-11-27 14:54:20,516 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:20,516 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:20,516 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:20,516 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:20,516 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,516 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:20,517 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:20,519 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:20,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:20,519 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:20,519 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,519 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:20,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:20,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:20,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:20,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59be6d83[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:20,522 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:20,522 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,523 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,523 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:20,523 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,523 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:20,523 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:20,523 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:20,526 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,526 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:20,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:20,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:20,526 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:20,526 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:20,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:20,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:20,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:20,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:20,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:20,530 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:20,530 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:20,530 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:20,530 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,530 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:20,532 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,532 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:20,532 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:20,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:20,533 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:20,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,536 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:20,539 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:20,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:20,539 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:20,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:20,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:20,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:20,540 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,541 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:20,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:20,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:20,541 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:20,641 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:20,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:20,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:20,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:20,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,654 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:20,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:20,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:20,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:20,655 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,655 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,656 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:20,656 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:20,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:20,657 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:20,657 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:20,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,658 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:20,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,659 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,659 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-27 14:54:20,659 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:20,660 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:20,661 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:20,661 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:20,661 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:20,661 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:20,661 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:20,661 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:20,661 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:21,086 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601018c5ddb77c30d4d4ba919ab8e18e7a090 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601018c5ddb77c30d4d4ba919ab8e18e7a090 2023-11-27 14:54:21,529 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/49f298a5a0bc434583d7f3259e3ff454 2023-11-27 14:54:21,555 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/49f298a5a0bc434583d7f3259e3ff454 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/49f298a5a0bc434583d7f3259e3ff454 2023-11-27 14:54:21,564 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/49f298a5a0bc434583d7f3259e3ff454, entries=1, sequenceid=29, filesize=5.0 K 2023-11-27 14:54:21,565 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 904ms, sequenceid=29, compaction requested=true 2023-11-27 14:54:21,566 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:21,566 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:21,566 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:21,573 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,573 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:21,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:21,574 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,574 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:21,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:21,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:21,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:21,576 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:21,576 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2452f7c3[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:21,576 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:21,576 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,577 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,578 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:21,578 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:21,578 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:21,578 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,578 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:21,579 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,579 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:21,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,579 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:21,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:21,579 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:21,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:21,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:21,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:21,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:21,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:21,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:21,583 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:21,583 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:21,583 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:21,583 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,583 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:21,590 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,590 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:21,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:21,590 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:21,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:21,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:21,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:21,590 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:21,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,592 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:21,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,595 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,595 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:21,595 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,595 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:21,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:21,595 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-27 14:54:21,595 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:21,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:21,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:21,595 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:21,595 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,596 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,596 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,596 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:21,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:21,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:21,596 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:21,697 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:21,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:21,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:21,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:21,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,708 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:21,708 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:21,708 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:21,708 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:21,709 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,709 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,711 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:21,711 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:21,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:21,711 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:21,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,711 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:21,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:21,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,713 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-27 14:54:21,713 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:21,713 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:21,713 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:21,713 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:21,714 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:21,714 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:21,714 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:21,714 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:21,714 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:21,734 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038a3afc1078614f3f838f6248176f6f89 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038a3afc1078614f3f838f6248176f6f89 2023-11-27 14:54:22,143 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/065c196a81784ea0ae747196d7c1566e 2023-11-27 14:54:22,153 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/065c196a81784ea0ae747196d7c1566e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/065c196a81784ea0ae747196d7c1566e 2023-11-27 14:54:22,160 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/065c196a81784ea0ae747196d7c1566e, entries=1, sequenceid=33, filesize=5.0 K 2023-11-27 14:54:22,161 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 447ms, sequenceid=33, compaction requested=true 2023-11-27 14:54:22,161 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:22,161 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:22,161 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:22,162 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:22,162 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,162 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:22,162 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:22,164 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,164 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:22,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:22,164 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,165 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:22,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:22,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:22,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:22,168 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:22,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@629b177a[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:22,169 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:22,169 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,170 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,170 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:22,170 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:22,170 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:22,171 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,171 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:22,172 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,172 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:22,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:22,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:22,172 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:22,172 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:22,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:22,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:22,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:22,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:22,177 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:22,177 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:22,177 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:22,177 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:22,177 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,177 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:22,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:22,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:22,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:22,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:22,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:22,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:22,182 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:22,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,183 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:22,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,186 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,186 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,187 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,187 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,187 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,187 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:22,187 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:22,187 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:22,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:22,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-27 14:54:22,187 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:22,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:22,187 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:22,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:22,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:22,188 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:22,188 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:22,288 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:22,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:22,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:22,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:22,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,300 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:22,300 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:22,301 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:22,301 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:22,301 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,302 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,303 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:22,303 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:22,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:22,303 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:22,304 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:22,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:22,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,305 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,305 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-27 14:54:22,305 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:22,305 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:22,305 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:22,305 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:22,306 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:22,306 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:22,306 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:22,306 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:22,306 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:22,727 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907996dfc0dca8f4f548454a3b5efc03e7b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907996dfc0dca8f4f548454a3b5efc03e7b 2023-11-27 14:54:23,138 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/a4575f2091f34cc6917a5138de16a449 2023-11-27 14:54:23,146 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/a4575f2091f34cc6917a5138de16a449 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/a4575f2091f34cc6917a5138de16a449 2023-11-27 14:54:23,153 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/a4575f2091f34cc6917a5138de16a449, entries=1, sequenceid=37, filesize=5.0 K 2023-11-27 14:54:23,154 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 848ms, sequenceid=37, compaction requested=true 2023-11-27 14:54:23,155 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:23,155 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:23,155 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:23,155 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:23,155 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,155 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:23,155 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:23,157 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,157 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:23,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:23,158 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,158 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:23,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:23,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:23,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63230434[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:23,160 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:23,160 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,162 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,162 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:23,162 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:23,162 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:23,162 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,162 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:23,164 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,164 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:23,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:23,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:23,164 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:23,164 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:23,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:23,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:23,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:23,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:23,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:23,169 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:23,169 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:23,169 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:23,169 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,169 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:23,171 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:23,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:23,172 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,172 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:23,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:23,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:23,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:23,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,173 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:23,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,174 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:23,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:23,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:23,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:23,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:23,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:23,181 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:23,181 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:23,182 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,182 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:23,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:23,182 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:23,183 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:23,283 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:23,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:23,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:23,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:23,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,295 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:23,295 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:23,295 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-27 14:54:23,295 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-27 14:54:23,296 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,296 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,297 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,297 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:23,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:23,298 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:23,298 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:23,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,299 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,299 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-27 14:54:23,299 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:23,300 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-27 14:54:23,300 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:23,301 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-27 14:54:23,301 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:23,301 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:23,301 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:23,301 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. started... 2023-11-27 14:54:23,301 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 73f02d3ec84561cf82fbd52d7e1039a0 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:23,725 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120040eeab0135042adb2142bb2a079e382 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120040eeab0135042adb2142bb2a079e382 2023-11-27 14:54:24,136 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/9c6edcd6cd8b491fb2887aceefccc0b9 2023-11-27 14:54:24,144 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/9c6edcd6cd8b491fb2887aceefccc0b9 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/9c6edcd6cd8b491fb2887aceefccc0b9 2023-11-27 14:54:24,151 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/9c6edcd6cd8b491fb2887aceefccc0b9, entries=1, sequenceid=41, filesize=5.0 K 2023-11-27 14:54:24,152 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73f02d3ec84561cf82fbd52d7e1039a0 in 851ms, sequenceid=41, compaction requested=true 2023-11-27 14:54:24,153 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:24,153 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:24,153 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:24,153 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:24,153 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,153 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-27 14:54:24,153 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-27 14:54:24,155 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,155 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:24,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:24,156 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,156 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:24,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:24,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:24,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:24,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-27 14:54:24,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@495bafa6[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:24,157 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-27 14:54:24,157 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,159 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,159 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-27 14:54:24,159 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-27 14:54:24,159 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,159 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:24,159 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:24,161 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,161 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-27 14:54:24,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:24,161 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:24,162 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-27 14:54:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:24,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:24,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-27 14:54:24,165 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-27 14:54:24,165 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:24,165 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:24,165 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,165 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:24,167 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,167 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:24,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:24,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,167 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:24,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:24,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:24,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:24,168 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:24,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,170 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:24,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,172 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,172 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:24,172 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,173 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:24,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-27 14:54:24,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:24,173 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:24,173 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:24,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:24,173 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:24,174 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,174 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,174 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:24,174 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:24,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:24,174 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:24,274 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:24,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:24,281 DEBUG [Listener at localhost/37439] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-27 14:54:24,284 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45668, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-27 14:54:24,286 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:24,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-27 14:54:24,301 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-27 14:54:24,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:24,302 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097464302 2023-11-27 14:54:24,306 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:54:24,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-11-27 14:54:24,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:24,312 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x3e655241 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:54:24,316 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@10b2888e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:54:24,318 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:24,797 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035e6570ac09234aa3ac4bc85d24874fa3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601035e6570ac09234aa3ac4bc85d24874fa3 2023-11-27 14:54:24,798 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:24,817 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:54:24,819 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51898, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:54:24,833 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:54:24,835 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45680, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:54:24,879 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a907e21036cd4cfcbdcf978f98573b0a first=Optional[r3] last=Optional[r6] 2023-11-27 14:54:24,892 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a907e21036cd4cfcbdcf978f98573b0a}] 2023-11-27 14:54:24,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a907e21036cd4cfcbdcf978f98573b0a for inclusion in 73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:24,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-27 14:54:24,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:54:24,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:24,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a907e21036cd4cfcbdcf978f98573b0a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__sgg84jnu23uas2bii5jgt64brd5mt2rt21g0o4q5q208ooe4lts2k0vo5nl2komv/family1/a907e21036cd4cfcbdcf978f98573b0a 2023-11-27 14:54:24,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__sgg84jnu23uas2bii5jgt64brd5mt2rt21g0o4q5q208ooe4lts2k0vo5nl2komv/family1/a907e21036cd4cfcbdcf978f98573b0a as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/0458b90feca74ec8877a1a6b1394bcfa_SeqId_44_ 2023-11-27 14:54:24,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__sgg84jnu23uas2bii5jgt64brd5mt2rt21g0o4q5q208ooe4lts2k0vo5nl2komv/family1/a907e21036cd4cfcbdcf978f98573b0a into 73f02d3ec84561cf82fbd52d7e1039a0/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/0458b90feca74ec8877a1a6b1394bcfa_SeqId_44_ - updating store file list. 2023-11-27 14:54:24,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/0458b90feca74ec8877a1a6b1394bcfa_SeqId_44_ into 73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:24,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__sgg84jnu23uas2bii5jgt64brd5mt2rt21g0o4q5q208ooe4lts2k0vo5nl2komv/family1/a907e21036cd4cfcbdcf978f98573b0a into 73f02d3ec84561cf82fbd52d7e1039a0/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/0458b90feca74ec8877a1a6b1394bcfa_SeqId_44_) 2023-11-27 14:54:24,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__sgg84jnu23uas2bii5jgt64brd5mt2rt21g0o4q5q208ooe4lts2k0vo5nl2komv/family1/a907e21036cd4cfcbdcf978f98573b0a 2023-11-27 14:54:24,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-27 14:54:24,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-27 14:54:24,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): 73f02d3ec84561cf82fbd52d7e1039a0/family1 is initiating minor compaction 2023-11-27 14:54:24,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1e4ecd17; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:24,946 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 73f02d3ec84561cf82fbd52d7e1039a0/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:24,947 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/8fbbe081374e45338de1e16bac00d2cd, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/aee7cafffd7f42c0937d4cbf2e2d5b6f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f853f0671298496fa14517c21650fef9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/d075b26184d040b69849713958106385, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/1d735ed356c840d1b833f45f822bf41b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/49f298a5a0bc434583d7f3259e3ff454, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/065c196a81784ea0ae747196d7c1566e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/a4575f2091f34cc6917a5138de16a449, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/9c6edcd6cd8b491fb2887aceefccc0b9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/0458b90feca74ec8877a1a6b1394bcfa_SeqId_44_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp, totalSize=50.2 K 2023-11-27 14:54:24,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 8fbbe081374e45338de1e16bac00d2cd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-27 14:54:24,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting aee7cafffd7f42c0937d4cbf2e2d5b6f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-27 14:54:24,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting f853f0671298496fa14517c21650fef9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-27 14:54:24,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting d075b26184d040b69849713958106385, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-27 14:54:24,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 1d735ed356c840d1b833f45f822bf41b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-27 14:54:24,948 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 49f298a5a0bc434583d7f3259e3ff454, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-27 14:54:24,948 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 065c196a81784ea0ae747196d7c1566e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-27 14:54:24,948 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting a4575f2091f34cc6917a5138de16a449, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-27 14:54:24,948 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 9c6edcd6cd8b491fb2887aceefccc0b9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-27 14:54:24,948 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 0458b90feca74ec8877a1a6b1394bcfa_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-27 14:54:24,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:24,963 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:24,968 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228f5a8e17fea4a4f0ba2c33c0c3efaae15 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228f5a8e17fea4a4f0ba2c33c0c3efaae15 2023-11-27 14:54:24,970 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512311f8c4587931b44f89e2df6ebac82168a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512311f8c4587931b44f89e2df6ebac82168a 2023-11-27 14:54:24,972 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601018c5ddb77c30d4d4ba919ab8e18e7a090 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601018c5ddb77c30d4d4ba919ab8e18e7a090 2023-11-27 14:54:24,974 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038a3afc1078614f3f838f6248176f6f89 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038a3afc1078614f3f838f6248176f6f89 2023-11-27 14:54:24,974 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-11-27 14:54:24,974 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-27 14:54:24,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:24,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:24,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:24,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:24,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:24,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:24,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:24,976 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:25,011 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 73f02d3ec84561cf82fbd52d7e1039a0#family1#compaction#10 average throughput is 0.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:54:25,490 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120576ccf46a794a4bb8800864eeb6ccd09f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120576ccf46a794a4bb8800864eeb6ccd09f 2023-11-27 14:54:25,492 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:25,523 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/46ad9861c159459e93d51df6bef5e0ac first=Optional[r00] last=Optional[r2] 2023-11-27 14:54:25,523 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/46ad9861c159459e93d51df6bef5e0ac}] 2023-11-27 14:54:25,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/46ad9861c159459e93d51df6bef5e0ac for inclusion in 73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:25,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-27 14:54:25,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:54:25,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:25,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/46ad9861c159459e93d51df6bef5e0ac to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tm95omg33to6t3a7h8oercdq5p8bj2rv87k5feq3ncrcd7khikdc843q50akag7v/family1/46ad9861c159459e93d51df6bef5e0ac 2023-11-27 14:54:25,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tm95omg33to6t3a7h8oercdq5p8bj2rv87k5feq3ncrcd7khikdc843q50akag7v/family1/46ad9861c159459e93d51df6bef5e0ac as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_ 2023-11-27 14:54:25,535 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tm95omg33to6t3a7h8oercdq5p8bj2rv87k5feq3ncrcd7khikdc843q50akag7v/family1/46ad9861c159459e93d51df6bef5e0ac into 73f02d3ec84561cf82fbd52d7e1039a0/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_ - updating store file list. 2023-11-27 14:54:25,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_ into 73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:25,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tm95omg33to6t3a7h8oercdq5p8bj2rv87k5feq3ncrcd7khikdc843q50akag7v/family1/46ad9861c159459e93d51df6bef5e0ac into 73f02d3ec84561cf82fbd52d7e1039a0/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_) 2023-11-27 14:54:25,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__tm95omg33to6t3a7h8oercdq5p8bj2rv87k5feq3ncrcd7khikdc843q50akag7v/family1/46ad9861c159459e93d51df6bef5e0ac 2023-11-27 14:54:25,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:25,546 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:25,548 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130bee1a9b42391452db2a7e5481c0cd989 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130bee1a9b42391452db2a7e5481c0cd989 2023-11-27 14:54:25,550 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512019d040883cbac41bda6ddfd047e26b631 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512019d040883cbac41bda6ddfd047e26b631 2023-11-27 14:54:25,552 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205f1737397861e4820a07cfc0c4bca858d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205f1737397861e4820a07cfc0c4bca858d 2023-11-27 14:54:25,552 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-11-27 14:54:25,552 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-27 14:54:25,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:25,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:25,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:25,554 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-27 14:54:25,556 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.2630 sec 2023-11-27 14:54:25,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-27 14:54:25,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:25,848 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/f00c58b00a87414bb3aa859e3521ca94 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f00c58b00a87414bb3aa859e3521ca94 2023-11-27 14:54:25,861 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 73f02d3ec84561cf82fbd52d7e1039a0/family1 of 73f02d3ec84561cf82fbd52d7e1039a0 into f00c58b00a87414bb3aa859e3521ca94(size=6.6 K), total size for store is 358.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:54:25,862 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:25,863 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1701096864944; duration=0sec 2023-11-27 14:54:25,864 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:25,963 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/8fbbe081374e45338de1e16bac00d2cd, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/aee7cafffd7f42c0937d4cbf2e2d5b6f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f853f0671298496fa14517c21650fef9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/d075b26184d040b69849713958106385, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/1d735ed356c840d1b833f45f822bf41b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/49f298a5a0bc434583d7f3259e3ff454, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/065c196a81784ea0ae747196d7c1566e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/a4575f2091f34cc6917a5138de16a449, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/9c6edcd6cd8b491fb2887aceefccc0b9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/0458b90feca74ec8877a1a6b1394bcfa_SeqId_44_] to archive 2023-11-27 14:54:25,964 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:25,967 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/8fbbe081374e45338de1e16bac00d2cd to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/8fbbe081374e45338de1e16bac00d2cd 2023-11-27 14:54:25,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/aee7cafffd7f42c0937d4cbf2e2d5b6f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/aee7cafffd7f42c0937d4cbf2e2d5b6f 2023-11-27 14:54:25,970 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f853f0671298496fa14517c21650fef9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f853f0671298496fa14517c21650fef9 2023-11-27 14:54:25,972 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/d075b26184d040b69849713958106385 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/d075b26184d040b69849713958106385 2023-11-27 14:54:25,973 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/1d735ed356c840d1b833f45f822bf41b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/1d735ed356c840d1b833f45f822bf41b 2023-11-27 14:54:25,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/49f298a5a0bc434583d7f3259e3ff454 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/49f298a5a0bc434583d7f3259e3ff454 2023-11-27 14:54:25,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/065c196a81784ea0ae747196d7c1566e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/065c196a81784ea0ae747196d7c1566e 2023-11-27 14:54:25,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/a4575f2091f34cc6917a5138de16a449 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/a4575f2091f34cc6917a5138de16a449 2023-11-27 14:54:25,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/9c6edcd6cd8b491fb2887aceefccc0b9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/9c6edcd6cd8b491fb2887aceefccc0b9 2023-11-27 14:54:25,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/0458b90feca74ec8877a1a6b1394bcfa_SeqId_44_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/0458b90feca74ec8877a1a6b1394bcfa_SeqId_44_ 2023-11-27 14:54:26,135 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-11-27 14:54:26,137 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-11-27 14:54:26,137 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-11-27 14:54:26,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-11-27 14:54:26,155 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=73f02d3ec84561cf82fbd52d7e1039a0, REOPEN/MOVE}] 2023-11-27 14:54:26,157 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=73f02d3ec84561cf82fbd52d7e1039a0, REOPEN/MOVE 2023-11-27 14:54:26,158 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=73f02d3ec84561cf82fbd52d7e1039a0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:26,158 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701096866158"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096866158"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096866158"}]},"ts":"1701096866158"} 2023-11-27 14:54:26,160 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 73f02d3ec84561cf82fbd52d7e1039a0, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:26,317 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:26,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:26,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 73f02d3ec84561cf82fbd52d7e1039a0, disabling compactions & flushes 2023-11-27 14:54:26,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:26,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:26,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. after waiting 0 ms 2023-11-27 14:54:26,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:26,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-11-27 14:54:26,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:26,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:26,330 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: 73f02d3ec84561cf82fbd52d7e1039a0 to self. 2023-11-27 14:54:26,333 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:26,333 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=73f02d3ec84561cf82fbd52d7e1039a0, regionState=CLOSED 2023-11-27 14:54:26,334 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701096866333"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096866333"}]},"ts":"1701096866333"} 2023-11-27 14:54:26,338 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=16, resume processing ppid=15 2023-11-27 14:54:26,338 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 73f02d3ec84561cf82fbd52d7e1039a0, server=jenkins-hbase4.apache.org,42669,1701096850360 in 176 msec 2023-11-27 14:54:26,339 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=73f02d3ec84561cf82fbd52d7e1039a0, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=true 2023-11-27 14:54:26,490 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=73f02d3ec84561cf82fbd52d7e1039a0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:26,490 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701096866489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096866489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096866489"}]},"ts":"1701096866489"} 2023-11-27 14:54:26,492 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 73f02d3ec84561cf82fbd52d7e1039a0, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:26,645 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:26,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:26,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 73f02d3ec84561cf82fbd52d7e1039a0, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:54:26,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:26,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:26,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:26,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:26,652 INFO [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:26,653 DEBUG [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:26,653 DEBUG [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:26,653 INFO [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73f02d3ec84561cf82fbd52d7e1039a0 columnFamilyName family1 2023-11-27 14:54:26,662 DEBUG [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_ 2023-11-27 14:54:26,668 DEBUG [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/c75b799ab0ba44ef935eae0b27030c9b 2023-11-27 14:54:26,673 DEBUG [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f00c58b00a87414bb3aa859e3521ca94 2023-11-27 14:54:26,673 INFO [StoreOpener-73f02d3ec84561cf82fbd52d7e1039a0-1] regionserver.HStore(310): Store=73f02d3ec84561cf82fbd52d7e1039a0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:26,675 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:26,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:26,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:54:26,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 73f02d3ec84561cf82fbd52d7e1039a0; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66830153, jitterRate=-0.004153117537498474}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:54:26,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:26,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0., pid=17, masterSystemTime=1701096866645 2023-11-27 14:54:26,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:26,683 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-27 14:54:26,685 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-27 14:54:26,685 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1912): 73f02d3ec84561cf82fbd52d7e1039a0/family1 is initiating minor compaction (all files) 2023-11-27 14:54:26,685 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 73f02d3ec84561cf82fbd52d7e1039a0/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:26,685 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/c75b799ab0ba44ef935eae0b27030c9b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f00c58b00a87414bb3aa859e3521ca94, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp, totalSize=358.6 K 2023-11-27 14:54:26,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:26,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:54:26,686 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting c75b799ab0ba44ef935eae0b27030c9b, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878813266 2023-11-27 14:54:26,686 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=73f02d3ec84561cf82fbd52d7e1039a0, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:26,686 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting f00c58b00a87414bb3aa859e3521ca94, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621213266 2023-11-27 14:54:26,687 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1701096866686"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096866686"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096866686"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096866686"}]},"ts":"1701096866686"} 2023-11-27 14:54:26,687 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878813266 2023-11-27 14:54:26,692 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=17, resume processing ppid=15 2023-11-27 14:54:26,692 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 73f02d3ec84561cf82fbd52d7e1039a0, server=jenkins-hbase4.apache.org,42669,1701096850360 in 197 msec 2023-11-27 14:54:26,694 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=15, resume processing ppid=14 2023-11-27 14:54:26,694 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=73f02d3ec84561cf82fbd52d7e1039a0, REOPEN/MOVE in 537 msec 2023-11-27 14:54:26,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=14, resume processing ppid=13 2023-11-27 14:54:26,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 555 msec 2023-11-27 14:54:26,701 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 985 msec 2023-11-27 14:54:26,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=13 2023-11-27 14:54:26,712 INFO [Listener at localhost/37439] compactions.TestMobCompactor(278): alter status finished 2023-11-27 14:54:26,713 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-27 14:54:26,715 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-27 14:54:26,721 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-27 14:54:26,722 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097466722 2023-11-27 14:54:26,722 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:26,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:54:26,726 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-11-27 14:54:26,726 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:26,728 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x587a02ee to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:54:26,728 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 73f02d3ec84561cf82fbd52d7e1039a0#family1#compaction#11 average throughput is 5.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:54:26,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7910c3a3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:54:26,750 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:26,751 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:26,751 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:26,751 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:26,751 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:27,195 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511284ee3418594f7434bb099ffae7e722340 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511284ee3418594f7434bb099ffae7e722340 2023-11-27 14:54:27,196 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:27,211 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:54:27,213 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51910, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:54:27,222 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:54:27,224 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45696, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:54:27,238 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c879a2b3d61e4a8290d0264e99a92381 first=Optional[r05] last=Optional[r8] 2023-11-27 14:54:27,238 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=50 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c879a2b3d61e4a8290d0264e99a92381}] 2023-11-27 14:54:27,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c879a2b3d61e4a8290d0264e99a92381 for inclusion in 73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:27,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-27 14:54:27,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:54:27,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:27,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c879a2b3d61e4a8290d0264e99a92381 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__g2dfmrkcmoo1fm65qg18l165d0m5nq5b2ua7697ok2nev5v73sek145dbg4t7cl2/family1/c879a2b3d61e4a8290d0264e99a92381 2023-11-27 14:54:27,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__g2dfmrkcmoo1fm65qg18l165d0m5nq5b2ua7697ok2nev5v73sek145dbg4t7cl2/family1/c879a2b3d61e4a8290d0264e99a92381 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/249aed114a19446f834b2d8aba7bc781_SeqId_52_ 2023-11-27 14:54:27,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__g2dfmrkcmoo1fm65qg18l165d0m5nq5b2ua7697ok2nev5v73sek145dbg4t7cl2/family1/c879a2b3d61e4a8290d0264e99a92381 into 73f02d3ec84561cf82fbd52d7e1039a0/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/249aed114a19446f834b2d8aba7bc781_SeqId_52_ - updating store file list. 2023-11-27 14:54:27,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/249aed114a19446f834b2d8aba7bc781_SeqId_52_ into 73f02d3ec84561cf82fbd52d7e1039a0/family1 2023-11-27 14:54:27,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__g2dfmrkcmoo1fm65qg18l165d0m5nq5b2ua7697ok2nev5v73sek145dbg4t7cl2/family1/c879a2b3d61e4a8290d0264e99a92381 into 73f02d3ec84561cf82fbd52d7e1039a0/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/249aed114a19446f834b2d8aba7bc781_SeqId_52_) 2023-11-27 14:54:27,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__g2dfmrkcmoo1fm65qg18l165d0m5nq5b2ua7697ok2nev5v73sek145dbg4t7cl2/family1/c879a2b3d61e4a8290d0264e99a92381 2023-11-27 14:54:27,268 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:27,269 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:27,272 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120040eeab0135042adb2142bb2a079e382 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120040eeab0135042adb2142bb2a079e382 2023-11-27 14:54:27,274 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285e4c283308754ac1b63fdb300a940e63 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511285e4c283308754ac1b63fdb300a940e63 2023-11-27 14:54:27,274 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-11-27 14:54:27,275 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-27 14:54:27,275 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:27,275 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:27,275 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-27 14:54:27,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-27 14:54:27,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:27,277 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-27 14:54:27,279 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 563 msec 2023-11-27 14:54:27,414 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=253 (was 215) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:49938 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/595f7a7b59104a6d86e0f54e8df53bc7 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: hconnection-0x653c2357-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61098@0x3e655241-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:49952 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61098@0x3e655241 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/888108569.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:49982 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21c2af5f-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:49956 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x21c2af5f-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:49978 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:50166 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61098@0x587a02ee-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21c2af5f-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:50028 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:50042 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:49972 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:50054 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21c2af5f-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:42669-shortCompactions-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:940) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:880) org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:863) org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:818) org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.finishClose(HFileWriterImpl.java:820) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.close(HFileWriterImpl.java:654) org.apache.hadoop.hbase.regionserver.StoreFileWriter.close(StoreFileWriter.java:359) org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.commitWriter(DefaultCompactor.java:89) org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.commitWriter(DefaultCompactor.java:42) org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:362) org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.compact(DefaultMobStoreCompactor.java:108) org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122) org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1473) org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2267) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:625) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:673) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61098@0x587a02ee-SendThread(127.0.0.1:61098) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: hconnection-0x653c2357-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:50026 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:50012 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:50150 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21c2af5f-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61098@0x3e655241-SendThread(127.0.0.1:61098) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-45464582_17 at /127.0.0.1:49906 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x653c2357-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61098@0x587a02ee sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/888108569.run(Unknown Source) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=619 (was 505) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=325 (was 337), ProcessCount=168 (was 170), AvailableMemoryMB=8426 (was 8450) 2023-11-27 14:54:27,427 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=253, OpenFileDescriptor=619, MaxFileDescriptor=60000, SystemLoadAverage=325, ProcessCount=168, AvailableMemoryMB=8426 2023-11-27 14:54:27,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-27 14:54:27,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:27,433 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:54:27,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-11-27 14:54:27,434 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:54:27,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-27 14:54:27,436 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:54:27,437 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a empty. 2023-11-27 14:54:27,437 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:54:27,437 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-11-27 14:54:27,450 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-27 14:54:27,452 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 355881cf77b1bf9999fb7e0bd0dc8e2a, NAME => 'testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:54:27,459 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:27,459 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 355881cf77b1bf9999fb7e0bd0dc8e2a, disabling compactions & flushes 2023-11-27 14:54:27,459 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:27,459 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:27,460 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. after waiting 0 ms 2023-11-27 14:54:27,460 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:27,460 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:27,460 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:27,463 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:54:27,464 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701096867464"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096867464"}]},"ts":"1701096867464"} 2023-11-27 14:54:27,466 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-27 14:54:27,467 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:54:27,468 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096867467"}]},"ts":"1701096867467"} 2023-11-27 14:54:27,469 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-11-27 14:54:27,474 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=355881cf77b1bf9999fb7e0bd0dc8e2a, ASSIGN}] 2023-11-27 14:54:27,475 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=355881cf77b1bf9999fb7e0bd0dc8e2a, ASSIGN 2023-11-27 14:54:27,476 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=355881cf77b1bf9999fb7e0bd0dc8e2a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:54:27,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-27 14:54:27,569 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/.tmp/family1/595f7a7b59104a6d86e0f54e8df53bc7 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/595f7a7b59104a6d86e0f54e8df53bc7 2023-11-27 14:54:27,576 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 73f02d3ec84561cf82fbd52d7e1039a0/family1 of 73f02d3ec84561cf82fbd52d7e1039a0 into 595f7a7b59104a6d86e0f54e8df53bc7(size=177.2 K), total size for store is 182.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:54:27,576 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:54:27,576 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0., storeName=73f02d3ec84561cf82fbd52d7e1039a0/family1, priority=13, startTime=1701096866682; duration=0sec 2023-11-27 14:54:27,576 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:27,627 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=355881cf77b1bf9999fb7e0bd0dc8e2a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:27,628 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701096867627"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096867627"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096867627"}]},"ts":"1701096867627"} 2023-11-27 14:54:27,630 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 355881cf77b1bf9999fb7e0bd0dc8e2a, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:27,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/c75b799ab0ba44ef935eae0b27030c9b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f00c58b00a87414bb3aa859e3521ca94, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_] to archive 2023-11-27 14:54:27,658 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:27,660 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/c75b799ab0ba44ef935eae0b27030c9b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/c75b799ab0ba44ef935eae0b27030c9b 2023-11-27 14:54:27,661 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f00c58b00a87414bb3aa859e3521ca94 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/f00c58b00a87414bb3aa859e3521ca94 2023-11-27 14:54:27,663 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/family1/3415d41160e24a3d8bfdbfab649d64b4_SeqId_46_ 2023-11-27 14:54:27,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-27 14:54:27,782 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:27,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:27,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 355881cf77b1bf9999fb7e0bd0dc8e2a, NAME => 'testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:54:27,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:54:27,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:27,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:54:27,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:54:27,789 INFO [StoreOpener-355881cf77b1bf9999fb7e0bd0dc8e2a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:54:27,790 DEBUG [StoreOpener-355881cf77b1bf9999fb7e0bd0dc8e2a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1 2023-11-27 14:54:27,791 DEBUG [StoreOpener-355881cf77b1bf9999fb7e0bd0dc8e2a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1 2023-11-27 14:54:27,791 INFO [StoreOpener-355881cf77b1bf9999fb7e0bd0dc8e2a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 355881cf77b1bf9999fb7e0bd0dc8e2a columnFamilyName family1 2023-11-27 14:54:27,793 INFO [StoreOpener-355881cf77b1bf9999fb7e0bd0dc8e2a-1] regionserver.HStore(310): Store=355881cf77b1bf9999fb7e0bd0dc8e2a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:27,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:54:27,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:54:27,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:54:27,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:27,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 355881cf77b1bf9999fb7e0bd0dc8e2a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61539627, jitterRate=-0.08298809826374054}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:54:27,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:27,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a., pid=21, masterSystemTime=1701096867782 2023-11-27 14:54:27,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:27,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:27,805 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=355881cf77b1bf9999fb7e0bd0dc8e2a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:27,805 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701096867805"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096867805"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096867805"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096867805"}]},"ts":"1701096867805"} 2023-11-27 14:54:27,809 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=21, resume processing ppid=20 2023-11-27 14:54:27,809 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 355881cf77b1bf9999fb7e0bd0dc8e2a, server=jenkins-hbase4.apache.org,42669,1701096850360 in 177 msec 2023-11-27 14:54:27,811 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=20, resume processing ppid=19 2023-11-27 14:54:27,812 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=355881cf77b1bf9999fb7e0bd0dc8e2a, ASSIGN in 336 msec 2023-11-27 14:54:27,812 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:54:27,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096867812"}]},"ts":"1701096867812"} 2023-11-27 14:54:27,814 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-11-27 14:54:27,817 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:54:27,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 387 msec 2023-11-27 14:54:28,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-27 14:54:28,039 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-11-27 14:54:28,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:28,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,082 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:28,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:28,083 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:28,083 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:28,084 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,084 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,085 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:28,085 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:28,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:28,086 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,086 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:28,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,087 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:28,088 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:28,088 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:28,092 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:28,092 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:28,092 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:28,092 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:28,093 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-27 14:54:28,522 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b4ac8efef6294cd795253b0ebd45abef to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130b4ac8efef6294cd795253b0ebd45abef 2023-11-27 14:54:28,526 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/12404d609bfe4672b94a3159be553c09 2023-11-27 14:54:28,532 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/12404d609bfe4672b94a3159be553c09 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/12404d609bfe4672b94a3159be553c09 2023-11-27 14:54:28,538 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/12404d609bfe4672b94a3159be553c09, entries=1000, sequenceid=5, filesize=166.0 K 2023-11-27 14:54:28,540 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 446ms, sequenceid=5, compaction requested=false 2023-11-27 14:54:28,540 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:28,541 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:28,541 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:28,541 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:28,541 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:28,541 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,541 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:28,541 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:28,544 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,544 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:28,544 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,544 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:28,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:28,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:28,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:28,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:28,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a84b5a1[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:28,546 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:28,546 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,547 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,547 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:28,548 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,548 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:28,548 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:28,548 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:28,549 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,549 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:28,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:28,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:28,549 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:28,549 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:28,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:28,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:28,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:28,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:28,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:28,553 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:28,553 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:28,553 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:28,553 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,553 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:28,554 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,554 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:28,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:28,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,555 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:28,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:28,555 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:28,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:28,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:28,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,557 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:28,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,569 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,569 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:28,569 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:28,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:28,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:28,569 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:28,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:28,569 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:28,569 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,570 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,570 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,570 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:28,570 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:28,570 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,570 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:28,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:28,671 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:28,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:28,678 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:28,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:28,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,680 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:28,680 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:28,681 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:28,681 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:28,682 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,682 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,683 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,683 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:28,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:28,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:28,683 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:28,683 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:28,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,684 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:28,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,685 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,685 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:54:28,685 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:28,685 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:28,685 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:28,686 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:28,686 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:28,686 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:28,686 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:28,686 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:28,686 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-27 14:54:29,102 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ddcf617477ef4b3a8ae5f9479bd7da11 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128ddcf617477ef4b3a8ae5f9479bd7da11 2023-11-27 14:54:29,508 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/237d357288984a4ba345c97990faba3d 2023-11-27 14:54:29,516 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/237d357288984a4ba345c97990faba3d as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/237d357288984a4ba345c97990faba3d 2023-11-27 14:54:29,522 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/237d357288984a4ba345c97990faba3d, entries=1, sequenceid=9, filesize=5.0 K 2023-11-27 14:54:29,523 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 837ms, sequenceid=9, compaction requested=false 2023-11-27 14:54:29,523 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:29,523 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:29,523 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:29,523 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:29,523 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,523 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:29,523 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:29,525 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,525 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:29,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:29,526 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:29,526 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:29,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:29,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:29,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:29,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@409e4d4c[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:29,528 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:29,528 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,529 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,529 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:29,529 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:29,529 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:29,529 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,529 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:29,531 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:29,531 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,531 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:29,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:29,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:29,531 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:29,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:29,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:29,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:29,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:29,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:29,534 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:29,535 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:29,535 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:29,535 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,535 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:29,536 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,536 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:29,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:29,536 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:29,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:29,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:29,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:29,537 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:29,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,541 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:29,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,544 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:29,544 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,544 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:29,544 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:29,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:29,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:29,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:29,544 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:29,544 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,545 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,545 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,545 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:29,545 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,545 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:29,545 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:29,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:29,645 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:29,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:29,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:29,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:29,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,653 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:29,653 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:29,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:29,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:29,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,654 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,656 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,656 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:29,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:29,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:29,656 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:29,656 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:29,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,657 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:29,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,657 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,657 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:54:29,658 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:29,658 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:29,658 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:29,658 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:29,658 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:29,658 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:29,658 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:29,658 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:29,659 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:30,074 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120180b1fd19248a493d9adbe84e625e2ae0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120180b1fd19248a493d9adbe84e625e2ae0 2023-11-27 14:54:30,481 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/34dbd903ec5c48fea96887073c56a233 2023-11-27 14:54:30,488 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/34dbd903ec5c48fea96887073c56a233 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/34dbd903ec5c48fea96887073c56a233 2023-11-27 14:54:30,494 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/34dbd903ec5c48fea96887073c56a233, entries=1, sequenceid=13, filesize=4.9 K 2023-11-27 14:54:30,495 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 836ms, sequenceid=13, compaction requested=true 2023-11-27 14:54:30,495 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:30,495 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:30,495 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:30,496 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:30,496 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,496 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:30,496 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:30,498 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,498 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:30,499 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,499 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:30,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:30,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:30,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:30,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3be7f3cf[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:30,501 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:30,501 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,502 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:30,502 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,502 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:30,504 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:30,504 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,504 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:30,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,504 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:30,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:30,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:30,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:30,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:30,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:30,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:30,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:30,508 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:30,508 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:30,508 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:30,508 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,508 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:30,510 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,510 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,510 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:30,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:30,511 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,512 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:30,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,516 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,516 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,516 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,516 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,516 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,516 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:30,516 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:30,516 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:30,516 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,516 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:30,516 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:54:30,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:30,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:30,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:30,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:30,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:30,518 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:30,518 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:30,618 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:30,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:30,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:30,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:30,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,627 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:30,627 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:30,628 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:30,628 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:30,628 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,628 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,630 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,630 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:30,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:30,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:30,630 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:30,630 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:30,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:30,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,632 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:54:30,632 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:30,632 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:30,632 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:30,632 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:30,635 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:30,635 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:30,635 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:30,635 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:30,635 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:31,053 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205958358f62fbc4ce9a39bf043fb6dce7f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205958358f62fbc4ce9a39bf043fb6dce7f 2023-11-27 14:54:31,459 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/f8ff84dbe76e4dd5861ad7662c7c6246 2023-11-27 14:54:31,465 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/f8ff84dbe76e4dd5861ad7662c7c6246 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f8ff84dbe76e4dd5861ad7662c7c6246 2023-11-27 14:54:31,471 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f8ff84dbe76e4dd5861ad7662c7c6246, entries=1, sequenceid=17, filesize=4.9 K 2023-11-27 14:54:31,472 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 837ms, sequenceid=17, compaction requested=true 2023-11-27 14:54:31,472 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:31,472 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:31,472 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:31,472 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:31,472 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,472 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:31,472 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:31,474 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,474 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:31,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:31,475 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,475 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:31,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:31,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:31,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:31,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:31,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fd55d8a[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:31,477 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:31,477 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,479 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,479 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:31,479 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:31,479 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:31,479 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,479 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:31,487 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:31,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:31,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:31,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:31,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:31,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:31,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:31,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:31,491 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:31,491 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:31,491 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:31,491 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,491 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:31,492 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,492 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:31,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:31,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:31,493 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,495 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:31,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,497 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,497 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:31,497 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,497 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,497 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:31,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:31,497 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:31,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:31,498 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,498 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:31,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:31,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:54:31,498 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,499 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,499 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:31,499 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:31,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:31,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:31,599 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:31,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:31,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:31,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:31,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,607 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:31,607 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:31,608 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:31,608 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:31,608 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,608 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,610 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,610 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:31,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:31,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:31,610 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:31,610 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:31,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:31,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,611 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:54:31,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:31,611 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:31,611 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:31,612 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:31,612 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:31,612 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:31,612 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:31,612 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:31,612 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:31,862 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:54:32,027 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228bb3ce2c9840944b5b621dc6bd3f21abe to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228bb3ce2c9840944b5b621dc6bd3f21abe 2023-11-27 14:54:32,433 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/ffd3cfc2b280467ab3fbde9076456175 2023-11-27 14:54:32,440 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/ffd3cfc2b280467ab3fbde9076456175 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/ffd3cfc2b280467ab3fbde9076456175 2023-11-27 14:54:32,446 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/ffd3cfc2b280467ab3fbde9076456175, entries=1, sequenceid=21, filesize=4.9 K 2023-11-27 14:54:32,447 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 835ms, sequenceid=21, compaction requested=true 2023-11-27 14:54:32,447 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:32,447 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:32,447 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:32,447 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:32,447 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,447 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:32,447 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:32,450 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,450 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:32,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:32,451 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,451 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:32,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:32,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:32,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:32,452 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:32,452 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7023dc6c[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:32,452 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:32,452 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,454 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,454 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:32,454 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:32,454 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,454 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:32,454 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:32,456 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,456 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:32,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:32,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:32,456 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:32,456 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:32,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:32,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:32,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:32,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:32,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:32,460 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:32,460 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:32,460 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:32,460 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,460 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:32,462 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,462 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:32,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:32,462 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:32,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:32,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:32,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:32,463 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:32,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,464 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:32,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,468 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:32,468 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,468 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,468 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,468 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,468 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:32,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:32,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:32,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:32,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:54:32,468 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:32,468 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:32,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:32,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:32,469 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,470 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,470 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:32,470 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:32,570 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:32,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:32,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:32,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:32,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,579 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:32,579 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:32,580 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:32,580 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:32,580 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,580 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,582 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:32,582 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:32,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:32,583 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:32,583 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,583 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:32,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,583 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,584 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:54:32,584 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:32,584 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:32,584 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:32,584 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:32,585 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:32,585 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:32,585 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:32,585 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:32,585 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:33,001 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512312418a439af2d4caa90d638587173f1a5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512312418a439af2d4caa90d638587173f1a5 2023-11-27 14:54:33,407 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/6500a7cf27324494bf008c7c02310102 2023-11-27 14:54:33,413 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/6500a7cf27324494bf008c7c02310102 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/6500a7cf27324494bf008c7c02310102 2023-11-27 14:54:33,418 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/6500a7cf27324494bf008c7c02310102, entries=1, sequenceid=25, filesize=4.9 K 2023-11-27 14:54:33,419 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 834ms, sequenceid=25, compaction requested=true 2023-11-27 14:54:33,419 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:33,419 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:33,419 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:33,419 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:33,419 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,419 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:33,419 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:33,421 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,421 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:33,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:33,422 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,422 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:33,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:33,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:33,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:33,423 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:33,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42e50b09[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:33,424 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:33,424 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,425 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,425 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:33,425 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:33,425 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:33,425 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,425 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:33,427 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,427 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:33,427 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:33,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:33,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:33,427 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:33,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:33,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:33,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:33,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:33,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:33,430 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:33,430 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:33,430 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:33,430 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,430 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:33,433 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,433 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:33,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,433 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:33,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:33,434 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,435 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:33,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,437 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,437 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,437 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,437 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:33,437 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,437 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:33,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:54:33,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:33,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:33,438 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:33,437 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:33,438 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,439 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,439 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:33,439 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:33,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:33,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:33,539 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:33,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:33,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:33,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:33,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,549 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:33,549 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:33,550 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:33,550 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:33,550 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,550 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,552 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:33,552 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:33,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:33,552 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:33,552 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:33,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,553 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:33,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,554 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:54:33,554 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:33,554 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:33,554 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:33,555 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:33,555 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:33,974 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101481cd73f8258435895afb451b22decf2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101481cd73f8258435895afb451b22decf2 2023-11-27 14:54:34,381 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/f1fa948f50bf4a0caa10489685154fbb 2023-11-27 14:54:34,387 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/f1fa948f50bf4a0caa10489685154fbb as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f1fa948f50bf4a0caa10489685154fbb 2023-11-27 14:54:34,393 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f1fa948f50bf4a0caa10489685154fbb, entries=1, sequenceid=29, filesize=4.9 K 2023-11-27 14:54:34,394 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 839ms, sequenceid=29, compaction requested=true 2023-11-27 14:54:34,394 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:34,394 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:34,395 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:34,395 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:34,395 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,395 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:34,395 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:34,405 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,405 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:34,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:34,406 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:34,406 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:34,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:34,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:34,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:34,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4618f111[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:34,407 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:34,407 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,408 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,409 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:34,409 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,409 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:34,409 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:34,409 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:34,410 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:34,410 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:34,410 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,410 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:34,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:34,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:34,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:34,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:34,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:34,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:34,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:34,414 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:34,414 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:34,414 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:34,414 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,414 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:34,417 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,417 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:34,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:34,418 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:34,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:34,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:34,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:34,418 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:34,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,420 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:34,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,423 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,423 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,423 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,423 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,423 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:34,423 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,423 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:34,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:54:34,423 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:34,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:34,423 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:34,424 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:34,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:34,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:34,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:34,424 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:34,424 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:34,524 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:34,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:34,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:34,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:34,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,535 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:34,535 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:34,535 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:34,535 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:34,536 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,536 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,537 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,537 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:34,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:34,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:34,538 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,538 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:34,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:34,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,539 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:54:34,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,539 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:34,539 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:34,540 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:34,540 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:34,540 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:34,540 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:34,540 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:34,540 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:34,957 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030f90e2346bb44548b5faa242f5b01026 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030f90e2346bb44548b5faa242f5b01026 2023-11-27 14:54:34,965 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/1b51cfac4b9b4a11b852ecb3539e76ba 2023-11-27 14:54:34,971 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/1b51cfac4b9b4a11b852ecb3539e76ba as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/1b51cfac4b9b4a11b852ecb3539e76ba 2023-11-27 14:54:34,975 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/1b51cfac4b9b4a11b852ecb3539e76ba, entries=1, sequenceid=33, filesize=4.9 K 2023-11-27 14:54:34,976 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 436ms, sequenceid=33, compaction requested=true 2023-11-27 14:54:34,977 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:34,977 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:34,977 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:34,977 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:34,977 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,977 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:34,977 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:34,979 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,979 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:34,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:34,979 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,979 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:34,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:34,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:34,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:34,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:34,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4324dd55[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:34,981 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:34,981 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,983 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,983 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:34,983 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:34,983 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:34,983 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,983 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:34,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,985 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:34,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:34,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:34,985 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:34,985 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:34,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:34,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:34,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:34,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:34,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:34,988 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:34,988 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:34,988 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:34,988 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,988 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:34,990 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,990 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:34,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:34,990 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:34,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:34,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:34,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:34,990 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:34,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,992 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:34,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,994 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:34,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:34,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:34,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:34,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:34,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:34,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:34,995 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:34,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,997 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,997 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:34,997 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:34,997 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:34,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:34,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:35,097 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:35,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:35,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:35,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:35,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,106 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:35,106 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:35,107 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:35,107 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:35,107 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,107 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,110 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:35,110 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:35,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:35,110 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,110 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:35,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:35,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,111 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,111 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:54:35,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,112 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:35,112 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:35,112 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:35,112 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:35,112 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:35,112 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:35,112 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:35,113 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:35,530 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090714b7e6707f904559b35fcaba6953fb3f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015090714b7e6707f904559b35fcaba6953fb3f 2023-11-27 14:54:35,936 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/8837ccb83f4149ee97ee80e4bdbc989e 2023-11-27 14:54:35,942 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/8837ccb83f4149ee97ee80e4bdbc989e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8837ccb83f4149ee97ee80e4bdbc989e 2023-11-27 14:54:35,947 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8837ccb83f4149ee97ee80e4bdbc989e, entries=1, sequenceid=37, filesize=4.9 K 2023-11-27 14:54:35,948 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 835ms, sequenceid=37, compaction requested=true 2023-11-27 14:54:35,948 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:35,949 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:35,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:35,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:35,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:35,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:35,951 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,951 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:35,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:35,951 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,951 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:35,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:35,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:35,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:35,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:35,952 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d8ab9e1[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:35,952 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:35,953 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,954 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:35,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:35,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:35,954 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,954 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:35,955 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,955 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:35,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:35,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:35,955 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:35,956 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:35,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:35,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:35,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:35,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:35,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:35,958 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:35,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:35,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:35,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,958 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:35,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:35,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:35,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:35,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:35,962 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:35,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:35,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:35,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,963 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:35,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:35,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:35,968 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:35,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:35,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:54:35,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:35,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:35,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:35,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:35,969 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,969 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,969 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:35,969 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:35,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:35,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:35,969 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:36,069 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:36,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:36,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:36,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:36,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,077 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:36,077 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:36,078 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:54:36,078 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:36,078 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,078 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,080 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:36,080 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:36,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:36,080 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,080 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:36,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:36,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:54:36,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,081 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:54:36,081 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:36,082 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:54:36,083 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:36,083 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:36,083 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:36,083 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. started... 2023-11-27 14:54:36,083 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 355881cf77b1bf9999fb7e0bd0dc8e2a 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:36,497 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511202f15a62f3f944e5283907e44c2a6428a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511202f15a62f3f944e5283907e44c2a6428a 2023-11-27 14:54:36,903 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/0640f36c675e4575a10f5c2e70413f89 2023-11-27 14:54:36,910 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/0640f36c675e4575a10f5c2e70413f89 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/0640f36c675e4575a10f5c2e70413f89 2023-11-27 14:54:36,916 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/0640f36c675e4575a10f5c2e70413f89, entries=1, sequenceid=41, filesize=4.9 K 2023-11-27 14:54:36,917 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 355881cf77b1bf9999fb7e0bd0dc8e2a in 834ms, sequenceid=41, compaction requested=true 2023-11-27 14:54:36,917 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:36,918 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:54:36,918 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:54:36,920 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,920 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:36,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:36,920 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,920 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:36,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:36,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:36,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:36,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:54:36,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@475d521a[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:36,922 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:36,922 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,924 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:36,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:54:36,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:36,924 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,924 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:36,925 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,925 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:54:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:36,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:36,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:54:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:36,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:36,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:36,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:36,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:36,928 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:54:36,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:36,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:36,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,928 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:36,930 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,930 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:36,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:36,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,930 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:36,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:36,931 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:36,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,932 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:36,935 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:36,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:36,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-27 14:54:36,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:36,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:36,935 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:36,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:37,036 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:37,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:37,038 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-11-27 14:54:37,040 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:54:37,040 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:54:37,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:37,042 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097477042 2023-11-27 14:54:37,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:54:37,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-11-27 14:54:37,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:54:37,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x36434290 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:54:37,052 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49319220, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:54:37,052 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:54:37,889 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030a0d3720cd4a4003b3a14580380a6d7c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030a0d3720cd4a4003b3a14580380a6d7c 2023-11-27 14:54:37,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:54:37,904 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:54:37,905 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47302, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:54:37,912 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:54:37,913 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48662, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:54:37,923 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/50715ad9076148c594715dc6f0707e03 first=Optional[r5] last=Optional[r6] 2023-11-27 14:54:37,924 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/50715ad9076148c594715dc6f0707e03}] 2023-11-27 14:54:37,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/50715ad9076148c594715dc6f0707e03 for inclusion in 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 2023-11-27 14:54:37,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-11-27 14:54:37,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:54:37,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:37,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/50715ad9076148c594715dc6f0707e03 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__8kps3r4ofmcqe155gjjqkeloqk1f1itv2th10hj796b62v11fp3e67mc4gelj521/family1/50715ad9076148c594715dc6f0707e03 2023-11-27 14:54:37,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__8kps3r4ofmcqe155gjjqkeloqk1f1itv2th10hj796b62v11fp3e67mc4gelj521/family1/50715ad9076148c594715dc6f0707e03 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/69fc1f4a206440aabd6373af0a75ded0_SeqId_44_ 2023-11-27 14:54:37,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__8kps3r4ofmcqe155gjjqkeloqk1f1itv2th10hj796b62v11fp3e67mc4gelj521/family1/50715ad9076148c594715dc6f0707e03 into 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/69fc1f4a206440aabd6373af0a75ded0_SeqId_44_ - updating store file list. 2023-11-27 14:54:37,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/69fc1f4a206440aabd6373af0a75ded0_SeqId_44_ into 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 2023-11-27 14:54:37,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__8kps3r4ofmcqe155gjjqkeloqk1f1itv2th10hj796b62v11fp3e67mc4gelj521/family1/50715ad9076148c594715dc6f0707e03 into 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/69fc1f4a206440aabd6373af0a75ded0_SeqId_44_) 2023-11-27 14:54:37,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__8kps3r4ofmcqe155gjjqkeloqk1f1itv2th10hj796b62v11fp3e67mc4gelj521/family1/50715ad9076148c594715dc6f0707e03 2023-11-27 14:54:37,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-27 14:54:37,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-27 14:54:37,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 is initiating minor compaction 2023-11-27 14:54:37,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@73fc4db4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:37,946 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 in testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:54:37,946 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/237d357288984a4ba345c97990faba3d, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/34dbd903ec5c48fea96887073c56a233, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f8ff84dbe76e4dd5861ad7662c7c6246, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/ffd3cfc2b280467ab3fbde9076456175, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/6500a7cf27324494bf008c7c02310102, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f1fa948f50bf4a0caa10489685154fbb, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/1b51cfac4b9b4a11b852ecb3539e76ba, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8837ccb83f4149ee97ee80e4bdbc989e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/0640f36c675e4575a10f5c2e70413f89, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/69fc1f4a206440aabd6373af0a75ded0_SeqId_44_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp, totalSize=49.7 K 2023-11-27 14:54:37,946 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 237d357288984a4ba345c97990faba3d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-27 14:54:37,946 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 34dbd903ec5c48fea96887073c56a233, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-27 14:54:37,946 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting f8ff84dbe76e4dd5861ad7662c7c6246, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-27 14:54:37,946 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting ffd3cfc2b280467ab3fbde9076456175, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-27 14:54:37,946 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 6500a7cf27324494bf008c7c02310102, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-27 14:54:37,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting f1fa948f50bf4a0caa10489685154fbb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-27 14:54:37,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 1b51cfac4b9b4a11b852ecb3539e76ba, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-27 14:54:37,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 8837ccb83f4149ee97ee80e4bdbc989e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-27 14:54:37,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 0640f36c675e4575a10f5c2e70413f89, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-27 14:54:37,947 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 69fc1f4a206440aabd6373af0a75ded0_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-27 14:54:37,948 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:54:37,949 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:37,951 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101481cd73f8258435895afb451b22decf2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101481cd73f8258435895afb451b22decf2 2023-11-27 14:54:37,952 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030f90e2346bb44548b5faa242f5b01026 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601030f90e2346bb44548b5faa242f5b01026 2023-11-27 14:54:37,952 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-11-27 14:54:37,952 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-11-27 14:54:37,953 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:54:37,953 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:54:37,991 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 355881cf77b1bf9999fb7e0bd0dc8e2a#family1#compaction#22 average throughput is 0.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:54:38,014 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311a50b4aeca994cf98985e590375dc245 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512311a50b4aeca994cf98985e590375dc245 2023-11-27 14:54:38,016 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:54:38,047 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/8612fc91508c45a38f842cf3fd9c100d first=Optional[r1] last=Optional[r4] 2023-11-27 14:54:38,047 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/8612fc91508c45a38f842cf3fd9c100d}] 2023-11-27 14:54:38,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/8612fc91508c45a38f842cf3fd9c100d for inclusion in 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 2023-11-27 14:54:38,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-11-27 14:54:38,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:54:38,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:38,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/8612fc91508c45a38f842cf3fd9c100d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ukarska1gnnb60qa8fidsqqm5ei210osru9b5v0qjrgfjjum001rn4e0j7l478j9/family1/8612fc91508c45a38f842cf3fd9c100d 2023-11-27 14:54:38,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ukarska1gnnb60qa8fidsqqm5ei210osru9b5v0qjrgfjjum001rn4e0j7l478j9/family1/8612fc91508c45a38f842cf3fd9c100d as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/3fb3d99b55d54ca0a1dd8105d457d926_SeqId_46_ 2023-11-27 14:54:38,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ukarska1gnnb60qa8fidsqqm5ei210osru9b5v0qjrgfjjum001rn4e0j7l478j9/family1/8612fc91508c45a38f842cf3fd9c100d into 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/3fb3d99b55d54ca0a1dd8105d457d926_SeqId_46_ - updating store file list. 2023-11-27 14:54:38,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/3fb3d99b55d54ca0a1dd8105d457d926_SeqId_46_ into 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 2023-11-27 14:54:38,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ukarska1gnnb60qa8fidsqqm5ei210osru9b5v0qjrgfjjum001rn4e0j7l478j9/family1/8612fc91508c45a38f842cf3fd9c100d into 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/3fb3d99b55d54ca0a1dd8105d457d926_SeqId_46_) 2023-11-27 14:54:38,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ukarska1gnnb60qa8fidsqqm5ei210osru9b5v0qjrgfjjum001rn4e0j7l478j9/family1/8612fc91508c45a38f842cf3fd9c100d 2023-11-27 14:54:38,065 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:54:38,066 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:38,067 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120180b1fd19248a493d9adbe84e625e2ae0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120180b1fd19248a493d9adbe84e625e2ae0 2023-11-27 14:54:38,069 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205958358f62fbc4ce9a39bf043fb6dce7f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205958358f62fbc4ce9a39bf043fb6dce7f 2023-11-27 14:54:38,070 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228bb3ce2c9840944b5b621dc6bd3f21abe to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228bb3ce2c9840944b5b621dc6bd3f21abe 2023-11-27 14:54:38,071 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512312418a439af2d4caa90d638587173f1a5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512312418a439af2d4caa90d638587173f1a5 2023-11-27 14:54:38,071 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-11-27 14:54:38,071 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-11-27 14:54:38,072 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:54:38,072 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:54:38,130 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f587c7b8b9884579bba5210236d4aefe to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130f587c7b8b9884579bba5210236d4aefe 2023-11-27 14:54:38,132 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:54:38,164 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c302ac40d3b64527b01900b7f9631a08 first=Optional[r00] last=Optional[r8] 2023-11-27 14:54:38,164 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c302ac40d3b64527b01900b7f9631a08}] 2023-11-27 14:54:38,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c302ac40d3b64527b01900b7f9631a08 for inclusion in 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 2023-11-27 14:54:38,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-11-27 14:54:38,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:54:38,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:38,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c302ac40d3b64527b01900b7f9631a08 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__d0sheog2ak7sejq6qa55le8dvd42bago2carfbr8qi772l39r1apevg1a3knf595/family1/c302ac40d3b64527b01900b7f9631a08 2023-11-27 14:54:38,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__d0sheog2ak7sejq6qa55le8dvd42bago2carfbr8qi772l39r1apevg1a3knf595/family1/c302ac40d3b64527b01900b7f9631a08 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8ce328ecb05d4c038ca8114cd08d2754_SeqId_48_ 2023-11-27 14:54:38,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__d0sheog2ak7sejq6qa55le8dvd42bago2carfbr8qi772l39r1apevg1a3knf595/family1/c302ac40d3b64527b01900b7f9631a08 into 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8ce328ecb05d4c038ca8114cd08d2754_SeqId_48_ - updating store file list. 2023-11-27 14:54:38,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8ce328ecb05d4c038ca8114cd08d2754_SeqId_48_ into 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 2023-11-27 14:54:38,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__d0sheog2ak7sejq6qa55le8dvd42bago2carfbr8qi772l39r1apevg1a3knf595/family1/c302ac40d3b64527b01900b7f9631a08 into 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8ce328ecb05d4c038ca8114cd08d2754_SeqId_48_) 2023-11-27 14:54:38,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__d0sheog2ak7sejq6qa55le8dvd42bago2carfbr8qi772l39r1apevg1a3knf595/family1/c302ac40d3b64527b01900b7f9631a08 2023-11-27 14:54:38,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 13 store files, 10 compacting, 2 eligible, 16 blocking 2023-11-27 14:54:38,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-27 14:54:38,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-27 14:54:38,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(415): Not compacting testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. because compaction request was cancelled 2023-11-27 14:54:38,187 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:54:38,187 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:38,189 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511202f15a62f3f944e5283907e44c2a6428a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511202f15a62f3f944e5283907e44c2a6428a 2023-11-27 14:54:38,191 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128ddcf617477ef4b3a8ae5f9479bd7da11 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128ddcf617477ef4b3a8ae5f9479bd7da11 2023-11-27 14:54:38,192 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130b4ac8efef6294cd795253b0ebd45abef to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130b4ac8efef6294cd795253b0ebd45abef 2023-11-27 14:54:38,192 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-11-27 14:54:38,192 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-27 14:54:38,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:54:38,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:54:38,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:54:38,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:54:38,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:38,194 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:54:38,196 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.1550 sec 2023-11-27 14:54:38,329 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=267 (was 253) Potentially hanging thread: hconnection-0x4d1326ca-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/cluster_e76484b8-e34a-cbd1-e5d0-706505e5be75/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60562 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60762 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60698 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60626 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60858 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60850 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d1326ca-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60644 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60592 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d1326ca-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61098@0x36434290-SendThread(127.0.0.1:61098) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:49978 [Waiting for operation #36] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/cluster_e76484b8-e34a-cbd1-e5d0-706505e5be75/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61098@0x36434290 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/888108569.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60800 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Thread-463 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: hconnection-0x4d1326ca-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60726 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60756 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60804 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d1326ca-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60646 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60730 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60766 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61098@0x36434290-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60802 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b84b1873761146b583906f5b88040f52 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60778 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60742 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60658 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d1326ca-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x697684f-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60712 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_548850199_17 at /127.0.0.1:60788 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d1326ca-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=658 (was 619) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=320 (was 325), ProcessCount=168 (was 168), AvailableMemoryMB=8413 (was 8426) 2023-11-27 14:54:38,342 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=267, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=320, ProcessCount=168, AvailableMemoryMB=8413 2023-11-27 14:54:38,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-27 14:54:38,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:38,348 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:54:38,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-11-27 14:54:38,349 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:54:38,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-27 14:54:38,350 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3 2023-11-27 14:54:38,351 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3 empty. 2023-11-27 14:54:38,351 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3 2023-11-27 14:54:38,351 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-11-27 14:54:38,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-27 14:54:38,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-27 14:54:38,761 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-27 14:54:38,763 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => b70da5c01d91e889a747889289d215f3, NAME => 'testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:54:38,812 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/.tmp/family1/df7a038a2b704ae08b53cd653d84b292 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/df7a038a2b704ae08b53cd653d84b292 2023-11-27 14:54:38,818 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 355881cf77b1bf9999fb7e0bd0dc8e2a/family1 of 355881cf77b1bf9999fb7e0bd0dc8e2a into df7a038a2b704ae08b53cd653d84b292(size=6.6 K), total size for store is 344.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:54:38,818 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:54:38,819 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1701096877945; duration=0sec 2023-11-27 14:54:38,819 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:38,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/237d357288984a4ba345c97990faba3d, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/34dbd903ec5c48fea96887073c56a233, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f8ff84dbe76e4dd5861ad7662c7c6246, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/ffd3cfc2b280467ab3fbde9076456175, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/6500a7cf27324494bf008c7c02310102, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f1fa948f50bf4a0caa10489685154fbb, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/1b51cfac4b9b4a11b852ecb3539e76ba, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8837ccb83f4149ee97ee80e4bdbc989e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/0640f36c675e4575a10f5c2e70413f89, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/69fc1f4a206440aabd6373af0a75ded0_SeqId_44_] to archive 2023-11-27 14:54:38,860 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:38,862 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/237d357288984a4ba345c97990faba3d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/237d357288984a4ba345c97990faba3d 2023-11-27 14:54:38,864 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/34dbd903ec5c48fea96887073c56a233 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/34dbd903ec5c48fea96887073c56a233 2023-11-27 14:54:38,865 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f8ff84dbe76e4dd5861ad7662c7c6246 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f8ff84dbe76e4dd5861ad7662c7c6246 2023-11-27 14:54:38,866 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/ffd3cfc2b280467ab3fbde9076456175 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/ffd3cfc2b280467ab3fbde9076456175 2023-11-27 14:54:38,868 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/6500a7cf27324494bf008c7c02310102 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/6500a7cf27324494bf008c7c02310102 2023-11-27 14:54:38,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f1fa948f50bf4a0caa10489685154fbb to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/f1fa948f50bf4a0caa10489685154fbb 2023-11-27 14:54:38,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/1b51cfac4b9b4a11b852ecb3539e76ba to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/1b51cfac4b9b4a11b852ecb3539e76ba 2023-11-27 14:54:38,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8837ccb83f4149ee97ee80e4bdbc989e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/8837ccb83f4149ee97ee80e4bdbc989e 2023-11-27 14:54:38,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/0640f36c675e4575a10f5c2e70413f89 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/0640f36c675e4575a10f5c2e70413f89 2023-11-27 14:54:38,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/69fc1f4a206440aabd6373af0a75ded0_SeqId_44_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/family1/69fc1f4a206440aabd6373af0a75ded0_SeqId_44_ 2023-11-27 14:54:38,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-27 14:54:39,170 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:39,170 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing b70da5c01d91e889a747889289d215f3, disabling compactions & flushes 2023-11-27 14:54:39,170 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:39,170 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:39,170 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. after waiting 0 ms 2023-11-27 14:54:39,170 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:39,170 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:39,170 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:39,173 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:54:39,174 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701096879174"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096879174"}]},"ts":"1701096879174"} 2023-11-27 14:54:39,176 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-27 14:54:39,177 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:54:39,178 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096879177"}]},"ts":"1701096879177"} 2023-11-27 14:54:39,179 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-11-27 14:54:39,183 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=b70da5c01d91e889a747889289d215f3, ASSIGN}] 2023-11-27 14:54:39,184 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=b70da5c01d91e889a747889289d215f3, ASSIGN 2023-11-27 14:54:39,185 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=b70da5c01d91e889a747889289d215f3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:54:39,336 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=b70da5c01d91e889a747889289d215f3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:39,337 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701096879336"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096879336"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096879336"}]},"ts":"1701096879336"} 2023-11-27 14:54:39,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure b70da5c01d91e889a747889289d215f3, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:39,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-27 14:54:39,491 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:39,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:39,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b70da5c01d91e889a747889289d215f3, NAME => 'testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:54:39,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy b70da5c01d91e889a747889289d215f3 2023-11-27 14:54:39,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:39,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b70da5c01d91e889a747889289d215f3 2023-11-27 14:54:39,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b70da5c01d91e889a747889289d215f3 2023-11-27 14:54:39,497 INFO [StoreOpener-b70da5c01d91e889a747889289d215f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b70da5c01d91e889a747889289d215f3 2023-11-27 14:54:39,499 DEBUG [StoreOpener-b70da5c01d91e889a747889289d215f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1 2023-11-27 14:54:39,499 DEBUG [StoreOpener-b70da5c01d91e889a747889289d215f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1 2023-11-27 14:54:39,499 INFO [StoreOpener-b70da5c01d91e889a747889289d215f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b70da5c01d91e889a747889289d215f3 columnFamilyName family1 2023-11-27 14:54:39,501 INFO [StoreOpener-b70da5c01d91e889a747889289d215f3-1] regionserver.HStore(310): Store=b70da5c01d91e889a747889289d215f3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:39,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3 2023-11-27 14:54:39,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3 2023-11-27 14:54:39,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b70da5c01d91e889a747889289d215f3 2023-11-27 14:54:39,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:39,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b70da5c01d91e889a747889289d215f3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72544577, jitterRate=0.08099843561649323}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:54:39,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:39,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3., pid=25, masterSystemTime=1701096879491 2023-11-27 14:54:39,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:39,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:39,511 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=b70da5c01d91e889a747889289d215f3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:39,511 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701096879511"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096879511"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096879511"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096879511"}]},"ts":"1701096879511"} 2023-11-27 14:54:39,514 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=25, resume processing ppid=24 2023-11-27 14:54:39,514 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure b70da5c01d91e889a747889289d215f3, server=jenkins-hbase4.apache.org,42669,1701096850360 in 173 msec 2023-11-27 14:54:39,516 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=24, resume processing ppid=23 2023-11-27 14:54:39,516 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=b70da5c01d91e889a747889289d215f3, ASSIGN in 332 msec 2023-11-27 14:54:39,517 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:54:39,517 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096879517"}]},"ts":"1701096879517"} 2023-11-27 14:54:39,519 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-11-27 14:54:39,521 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:54:39,523 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1760 sec 2023-11-27 14:54:40,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-27 14:54:40,455 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-11-27 14:54:40,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:40,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:40,481 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:40,481 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:40,481 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:40,481 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:40,482 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:40,482 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:40,484 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:40,484 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:40,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:40,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:40,484 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:40,484 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:40,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:40,485 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:40,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:40,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:40,485 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:40,485 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:40,486 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:40,486 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:40,486 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:40,486 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:40,486 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:40,486 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:40,486 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:40,487 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-27 14:54:40,909 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307f1830c2cd684d688bddb18e00ccd6c9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511307f1830c2cd684d688bddb18e00ccd6c9 2023-11-27 14:54:41,313 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/bafa20afd048478292c655ae3dda41f4 2023-11-27 14:54:41,320 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/bafa20afd048478292c655ae3dda41f4 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/bafa20afd048478292c655ae3dda41f4 2023-11-27 14:54:41,326 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/bafa20afd048478292c655ae3dda41f4, entries=1000, sequenceid=5, filesize=165.1 K 2023-11-27 14:54:41,327 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 840ms, sequenceid=5, compaction requested=false 2023-11-27 14:54:41,327 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:41,329 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:41,329 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:41,329 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:41,329 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:41,329 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,329 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:41,329 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:41,331 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,331 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:41,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:41,331 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,331 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:41,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:41,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:41,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:41,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:41,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46cca399[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:41,333 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:41,333 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,334 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,335 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:41,335 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:41,335 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:41,335 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,335 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:41,336 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:41,336 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,336 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:41,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:41,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:41,336 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:41,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:41,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:41,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:41,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:41,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:41,339 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:41,339 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:41,339 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:41,339 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,339 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:41,341 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:41,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:41,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:41,342 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,344 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:41,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,346 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,346 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:41,346 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:41,346 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:54:41,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:41,346 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:41,346 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:41,346 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:41,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:41,347 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,347 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,347 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,347 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,347 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,347 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:41,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:41,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:41,347 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:41,448 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:41,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:41,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:41,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:41,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,455 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:41,455 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:41,455 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:41,455 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:41,456 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,456 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,457 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,457 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:41,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:41,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:41,458 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:41,458 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:41,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,458 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:41,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,459 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,459 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:41,459 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:41,459 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:41,459 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:41,459 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:41,460 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:41,460 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:41,460 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:41,460 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:41,460 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-27 14:54:41,875 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112869ba4137ddfe458d8dbbfeebf662dbef to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015112869ba4137ddfe458d8dbbfeebf662dbef 2023-11-27 14:54:42,281 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/7b6782a3818044af98d7617198b9c7e4 2023-11-27 14:54:42,287 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/7b6782a3818044af98d7617198b9c7e4 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/7b6782a3818044af98d7617198b9c7e4 2023-11-27 14:54:42,292 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/7b6782a3818044af98d7617198b9c7e4, entries=1, sequenceid=9, filesize=5.0 K 2023-11-27 14:54:42,293 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 833ms, sequenceid=9, compaction requested=false 2023-11-27 14:54:42,293 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:42,293 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:42,293 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:42,293 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:42,293 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,294 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:42,294 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:42,300 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,300 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:42,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:42,301 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:42,301 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:42,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:42,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:42,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:42,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23e70060[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:42,303 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:42,303 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,304 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,304 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:42,304 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:42,304 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:42,304 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,304 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:42,306 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,306 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:42,306 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:42,306 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:42,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:42,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:42,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:42,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:42,308 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:42,308 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:42,308 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:42,309 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,309 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:42,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:42,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:42,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:42,311 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:42,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:42,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:42,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:42,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,312 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:42,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:42,316 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:42,316 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:42,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:42,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:42,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:42,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:42,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:42,316 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:42,417 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:42,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:42,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:42,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:42,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,425 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:42,425 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:42,425 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:42,425 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:42,425 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,425 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,427 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:42,427 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:42,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:42,427 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:42,427 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:42,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:42,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,428 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:42,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:42,429 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:42,429 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:42,429 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:42,429 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:42,429 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:42,429 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:42,429 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:42,429 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:42,846 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512012bef2b7045684945a41eae549cefd967 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512012bef2b7045684945a41eae549cefd967 2023-11-27 14:54:43,253 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/c3d859c50b5b42608ae34600f3535ce1 2023-11-27 14:54:43,259 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/c3d859c50b5b42608ae34600f3535ce1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c3d859c50b5b42608ae34600f3535ce1 2023-11-27 14:54:43,263 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c3d859c50b5b42608ae34600f3535ce1, entries=1, sequenceid=13, filesize=4.9 K 2023-11-27 14:54:43,264 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 835ms, sequenceid=13, compaction requested=true 2023-11-27 14:54:43,264 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:43,264 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:43,264 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:43,265 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:43,265 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,265 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:43,265 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:43,266 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,266 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:43,267 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:43,267 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:43,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:43,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b5af04c[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:43,268 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:43,269 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,270 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:43,270 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,270 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:43,272 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:43,272 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:43,272 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,272 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:43,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:43,275 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:43,275 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:43,275 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:43,275 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,275 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:43,277 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,277 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,277 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:43,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:43,278 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,279 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,281 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,281 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,281 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:43,281 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:43,281 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:43,281 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:43,281 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:43,281 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,281 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:54:43,282 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,281 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:43,282 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:43,282 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:43,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:43,382 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:43,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:43,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:43,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:43,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,390 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:43,390 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:43,390 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:43,390 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:43,391 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,391 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,392 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:43,392 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:43,392 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:43,393 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:43,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,393 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:43,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,393 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,393 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:43,394 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:43,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:43,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:43,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:43,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:43,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:43,394 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:43,394 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:43,394 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:43,808 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120567691713b3fa4ad296fac88025bbdeb3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120567691713b3fa4ad296fac88025bbdeb3 2023-11-27 14:54:44,215 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/36f0b1c75aac419bad50b06507ccfc9b 2023-11-27 14:54:44,221 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/36f0b1c75aac419bad50b06507ccfc9b as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/36f0b1c75aac419bad50b06507ccfc9b 2023-11-27 14:54:44,226 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/36f0b1c75aac419bad50b06507ccfc9b, entries=1, sequenceid=17, filesize=4.9 K 2023-11-27 14:54:44,227 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 833ms, sequenceid=17, compaction requested=true 2023-11-27 14:54:44,227 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:44,227 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:44,227 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:44,227 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:44,227 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,227 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:44,227 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:44,229 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,229 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:44,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:44,230 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,230 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:44,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:44,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:44,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:44,231 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:44,231 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76baa4bf[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:44,231 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:44,231 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,233 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,233 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:44,233 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:44,233 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:44,233 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,233 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:44,235 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,235 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:44,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:44,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:44,236 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:44,236 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:44,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:44,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:44,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:44,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:44,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:44,239 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:44,239 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:44,239 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:44,239 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,239 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:44,240 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,240 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:44,241 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:44,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:44,241 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,243 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:44,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:44,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:44,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:44,249 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:44,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,249 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:44,350 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:44,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:44,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:44,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:44,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,358 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:44,358 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:44,358 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:44,358 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:44,358 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,358 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,360 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,360 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:44,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:44,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:44,361 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:44,361 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:44,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:44,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,362 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:44,362 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:44,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:44,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:44,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:44,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:44,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:44,362 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:44,362 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:44,362 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:44,777 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512288d2a0ad5f0e34bc4a837442b699f4768 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512288d2a0ad5f0e34bc4a837442b699f4768 2023-11-27 14:54:45,033 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:54:45,183 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/0d72b1c0e9b24d21a7c77fc174049011 2023-11-27 14:54:45,189 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/0d72b1c0e9b24d21a7c77fc174049011 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/0d72b1c0e9b24d21a7c77fc174049011 2023-11-27 14:54:45,194 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/0d72b1c0e9b24d21a7c77fc174049011, entries=1, sequenceid=21, filesize=4.9 K 2023-11-27 14:54:45,195 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 833ms, sequenceid=21, compaction requested=true 2023-11-27 14:54:45,195 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:45,195 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:45,195 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:45,195 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:45,195 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,195 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:45,195 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:45,197 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,197 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:45,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:45,198 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,198 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:45,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:45,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:45,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55c9019a[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:45,199 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:45,199 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,200 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,201 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:45,201 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:45,201 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:45,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:45,202 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:45,202 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,202 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,202 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:45,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:45,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:45,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:45,205 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:45,205 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:45,205 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:45,205 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,205 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:45,210 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,210 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:45,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,210 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:45,211 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:45,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:45,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,213 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:45,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,215 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:45,216 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:54:45,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:45,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:45,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:45,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:45,216 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:45,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:45,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:45,317 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:45,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:45,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:45,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:45,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,324 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:45,324 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:45,324 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:45,324 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:45,324 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,324 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,326 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,326 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:45,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:45,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:45,326 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,326 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:45,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:45,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:45,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,327 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:45,328 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:45,328 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:45,328 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:45,328 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:45,328 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:45,328 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:45,328 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:45,743 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231b34e754042cb4bda84abd5b654c93972 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231b34e754042cb4bda84abd5b654c93972 2023-11-27 14:54:45,748 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/30b1dbed3d7a4b4b8e93aa7b794dc887 2023-11-27 14:54:45,753 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/30b1dbed3d7a4b4b8e93aa7b794dc887 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/30b1dbed3d7a4b4b8e93aa7b794dc887 2023-11-27 14:54:45,758 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/30b1dbed3d7a4b4b8e93aa7b794dc887, entries=1, sequenceid=25, filesize=4.9 K 2023-11-27 14:54:45,759 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 431ms, sequenceid=25, compaction requested=true 2023-11-27 14:54:45,759 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:45,759 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:45,759 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:45,759 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:45,759 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,759 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:45,759 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:45,761 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,761 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:45,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:45,762 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,762 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:45,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:45,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:45,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e36ef43[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:45,763 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:45,763 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,767 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,767 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:45,767 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:45,767 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,767 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:45,767 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:45,769 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,769 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:45,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:45,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:45,769 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:45,769 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:45,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:45,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:45,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:45,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:45,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:45,771 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:45,771 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:45,771 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:45,771 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,771 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:45,773 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,773 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:45,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:45,773 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:45,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:45,774 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:45,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:45,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:45,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,775 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:45,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,778 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,778 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,778 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,778 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:45,778 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:45,778 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:45,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:45,778 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,778 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,778 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,778 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:45,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:45,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:45,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:45,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:45,779 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:45,779 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:45,879 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:45,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:45,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:45,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:45,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,886 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:45,886 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:45,886 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:45,886 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:45,887 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,887 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,889 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,889 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:45,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:45,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:45,889 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:45,890 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:45,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:45,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,890 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,891 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:45,891 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:45,891 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:45,891 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:45,891 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:45,892 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:45,892 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:45,892 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:45,892 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:45,892 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:46,307 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ce70132a4b2044b5b0b9b3473621670e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101ce70132a4b2044b5b0b9b3473621670e 2023-11-27 14:54:46,714 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/aafbe6433db344b793e02c4c8ffd8df0 2023-11-27 14:54:46,720 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/aafbe6433db344b793e02c4c8ffd8df0 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/aafbe6433db344b793e02c4c8ffd8df0 2023-11-27 14:54:46,725 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/aafbe6433db344b793e02c4c8ffd8df0, entries=1, sequenceid=29, filesize=4.9 K 2023-11-27 14:54:46,726 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 834ms, sequenceid=29, compaction requested=true 2023-11-27 14:54:46,726 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:46,726 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:46,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:46,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:46,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:46,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:46,728 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,728 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:46,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:46,729 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,729 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:46,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:46,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:46,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:46,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:46,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42e2ae59[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:46,730 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:46,730 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,731 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,731 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:46,731 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:46,731 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:46,732 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,732 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:46,733 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,733 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:46,733 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:46,733 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:46,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:46,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:46,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:46,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:46,736 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:46,736 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:46,736 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:46,736 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,736 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:46,740 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:46,740 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,741 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:46,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:46,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:46,741 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,743 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,745 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,745 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:46,745 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,745 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:46,745 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:46,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:46,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:46,745 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:46,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:46,745 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,746 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:46,746 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:46,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:46,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:46,846 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:46,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:46,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:46,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:46,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,853 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:46,853 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:46,853 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:46,853 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:46,854 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,854 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,855 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,855 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:46,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:46,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:46,856 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:46,856 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:46,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,857 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,857 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:46,857 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:46,857 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:46,857 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:46,857 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:47,275 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103184af19c8d794edb8e84878ce082cb48 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103184af19c8d794edb8e84878ce082cb48 2023-11-27 14:54:47,282 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/5846e05a6b41463bbd8815264fe57a05 2023-11-27 14:54:47,287 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/5846e05a6b41463bbd8815264fe57a05 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5846e05a6b41463bbd8815264fe57a05 2023-11-27 14:54:47,291 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5846e05a6b41463bbd8815264fe57a05, entries=1, sequenceid=33, filesize=4.9 K 2023-11-27 14:54:47,292 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 435ms, sequenceid=33, compaction requested=true 2023-11-27 14:54:47,292 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:47,292 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:47,292 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:47,292 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:47,292 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,292 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:47,292 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:47,294 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,294 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:47,294 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,294 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:47,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:47,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@615e65e6[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:47,296 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:47,296 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,299 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:47,299 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:47,299 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,299 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:47,300 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:47,301 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:47,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:47,303 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:47,304 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:47,304 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:47,304 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,304 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:47,305 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,305 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:47,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:47,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,305 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:47,306 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,307 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:47,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:47,310 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:47,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:47,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:47,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:47,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:54:47,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:47,310 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,311 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:47,311 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:47,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:47,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:47,411 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:47,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:47,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:47,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:47,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,418 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:47,418 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:47,418 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:47,418 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:47,419 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,419 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,421 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,421 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:47,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:47,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:47,422 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:47,422 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:47,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,422 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:47,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:47,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:47,423 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:47,423 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:47,423 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:47,423 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:47,423 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:47,423 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:47,423 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:47,423 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:47,839 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907aae64edf55394c8baa8076102c434dcc to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907aae64edf55394c8baa8076102c434dcc 2023-11-27 14:54:48,245 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/ae94e7f56bfd47138b40370b8d65cc48 2023-11-27 14:54:48,251 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/ae94e7f56bfd47138b40370b8d65cc48 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/ae94e7f56bfd47138b40370b8d65cc48 2023-11-27 14:54:48,256 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/ae94e7f56bfd47138b40370b8d65cc48, entries=1, sequenceid=37, filesize=4.9 K 2023-11-27 14:54:48,257 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 834ms, sequenceid=37, compaction requested=true 2023-11-27 14:54:48,257 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:48,257 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:48,257 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:48,257 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:48,257 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,257 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:48,257 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:48,259 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,259 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:48,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:48,260 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,260 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:48,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:48,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:48,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:48,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:48,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c867d86[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:48,262 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:48,262 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,263 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,263 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:48,263 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:48,263 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:48,263 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,263 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:48,264 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,264 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:48,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:48,265 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:48,265 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:48,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:48,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:48,267 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:48,267 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:48,267 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:48,267 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:48,267 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,267 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:48,270 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,270 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,270 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:48,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:48,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:48,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:48,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:48,270 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:48,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:48,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,271 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:48,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,274 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:48,274 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,274 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:48,274 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:54:48,274 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,274 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:48,274 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:48,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:48,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:48,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:48,274 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,275 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,275 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,275 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:48,275 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:48,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:48,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:48,375 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:48,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:48,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:48,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:48,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,382 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:48,382 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:48,382 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:48,382 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:48,383 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,383 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,384 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,384 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:48,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:48,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:48,384 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:48,385 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,385 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,385 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,386 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:48,386 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:48,386 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:48,386 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:48,386 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:48,386 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:48,386 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:48,386 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:48,386 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. started... 2023-11-27 14:54:48,386 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b70da5c01d91e889a747889289d215f3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:48,800 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120fcacd92ed532474b852345a27c66f929 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120fcacd92ed532474b852345a27c66f929 2023-11-27 14:54:49,206 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/2fd9d5d192d44c6790eef7f41318ad02 2023-11-27 14:54:49,212 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/2fd9d5d192d44c6790eef7f41318ad02 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/2fd9d5d192d44c6790eef7f41318ad02 2023-11-27 14:54:49,216 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/2fd9d5d192d44c6790eef7f41318ad02, entries=1, sequenceid=41, filesize=4.9 K 2023-11-27 14:54:49,217 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b70da5c01d91e889a747889289d215f3 in 831ms, sequenceid=41, compaction requested=true 2023-11-27 14:54:49,217 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:49,217 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:49,217 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:49,217 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:49,217 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,217 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:49,217 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:49,219 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,219 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:49,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:49,220 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,220 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:49,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:49,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:49,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:49,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:49,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@533c9a2[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:49,221 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:49,221 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,225 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,225 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:49,225 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:49,225 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:49,225 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,225 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:49,227 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,227 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:49,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:49,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:49,227 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:49,228 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:49,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:49,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:49,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:49,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:49,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:49,230 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:49,230 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:49,230 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:49,230 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,230 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:49,232 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,232 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,232 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:49,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:49,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:49,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,232 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:49,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:49,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:49,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:49,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,234 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:49,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,236 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,236 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,236 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:49,236 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:49,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-27 14:54:49,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:49,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:49,236 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:49,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:54:49,236 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:49,236 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:49,237 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:49,237 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:49,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:49,237 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:49,237 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,237 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,337 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:49,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:49,340 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-11-27 14:54:49,342 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:54:49,342 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:54:49,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:49,343 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097489343 2023-11-27 14:54:49,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-27 14:54:49,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-11-27 14:54:49,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:54:49,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x0073972b to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:54:49,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5482d499, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:54:49,353 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:50,203 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d480fe1e23ca47c1b8f08d5a747e4f40 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103d480fe1e23ca47c1b8f08d5a747e4f40 2023-11-27 14:54:50,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:54:50,218 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:54:50,220 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44216, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:54:50,227 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:54:50,229 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38228, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:54:50,238 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ae05b5aa6dea447cbbdce45762c8662d first=Optional[r3] last=Optional[r6] 2023-11-27 14:54:50,239 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ae05b5aa6dea447cbbdce45762c8662d}] 2023-11-27 14:54:50,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ae05b5aa6dea447cbbdce45762c8662d for inclusion in b70da5c01d91e889a747889289d215f3/family1 2023-11-27 14:54:50,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-27 14:54:50,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:54:50,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:50,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ae05b5aa6dea447cbbdce45762c8662d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mmj4tenavleae72iprfg6o3utlng6hfbv8ji9ovjgufk8vloh998sghg6410sqi4/family1/ae05b5aa6dea447cbbdce45762c8662d 2023-11-27 14:54:50,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mmj4tenavleae72iprfg6o3utlng6hfbv8ji9ovjgufk8vloh998sghg6410sqi4/family1/ae05b5aa6dea447cbbdce45762c8662d as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c37510aeb8ba4af1b7a7e55151f036f5_SeqId_44_ 2023-11-27 14:54:50,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mmj4tenavleae72iprfg6o3utlng6hfbv8ji9ovjgufk8vloh998sghg6410sqi4/family1/ae05b5aa6dea447cbbdce45762c8662d into b70da5c01d91e889a747889289d215f3/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c37510aeb8ba4af1b7a7e55151f036f5_SeqId_44_ - updating store file list. 2023-11-27 14:54:50,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c37510aeb8ba4af1b7a7e55151f036f5_SeqId_44_ into b70da5c01d91e889a747889289d215f3/family1 2023-11-27 14:54:50,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mmj4tenavleae72iprfg6o3utlng6hfbv8ji9ovjgufk8vloh998sghg6410sqi4/family1/ae05b5aa6dea447cbbdce45762c8662d into b70da5c01d91e889a747889289d215f3/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c37510aeb8ba4af1b7a7e55151f036f5_SeqId_44_) 2023-11-27 14:54:50,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mmj4tenavleae72iprfg6o3utlng6hfbv8ji9ovjgufk8vloh998sghg6410sqi4/family1/ae05b5aa6dea447cbbdce45762c8662d 2023-11-27 14:54:50,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-27 14:54:50,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-27 14:54:50,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): b70da5c01d91e889a747889289d215f3/family1 is initiating minor compaction 2023-11-27 14:54:50,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3025a5fb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:50,261 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b70da5c01d91e889a747889289d215f3/family1 in testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:50,261 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/7b6782a3818044af98d7617198b9c7e4, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c3d859c50b5b42608ae34600f3535ce1, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/36f0b1c75aac419bad50b06507ccfc9b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/0d72b1c0e9b24d21a7c77fc174049011, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/30b1dbed3d7a4b4b8e93aa7b794dc887, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/aafbe6433db344b793e02c4c8ffd8df0, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5846e05a6b41463bbd8815264fe57a05, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/ae94e7f56bfd47138b40370b8d65cc48, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/2fd9d5d192d44c6790eef7f41318ad02, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c37510aeb8ba4af1b7a7e55151f036f5_SeqId_44_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp, totalSize=50.0 K 2023-11-27 14:54:50,261 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 7b6782a3818044af98d7617198b9c7e4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-27 14:54:50,262 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting c3d859c50b5b42608ae34600f3535ce1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-27 14:54:50,262 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 36f0b1c75aac419bad50b06507ccfc9b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-27 14:54:50,262 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 0d72b1c0e9b24d21a7c77fc174049011, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-27 14:54:50,262 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 30b1dbed3d7a4b4b8e93aa7b794dc887, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-27 14:54:50,262 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting aafbe6433db344b793e02c4c8ffd8df0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-27 14:54:50,262 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 5846e05a6b41463bbd8815264fe57a05, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-27 14:54:50,262 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting ae94e7f56bfd47138b40370b8d65cc48, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-27 14:54:50,262 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 2fd9d5d192d44c6790eef7f41318ad02, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-27 14:54:50,262 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting c37510aeb8ba4af1b7a7e55151f036f5_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-27 14:54:50,263 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:54:50,265 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:50,267 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512288d2a0ad5f0e34bc4a837442b699f4768 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512288d2a0ad5f0e34bc4a837442b699f4768 2023-11-27 14:54:50,268 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231b34e754042cb4bda84abd5b654c93972 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231b34e754042cb4bda84abd5b654c93972 2023-11-27 14:54:50,270 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101ce70132a4b2044b5b0b9b3473621670e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101ce70132a4b2044b5b0b9b3473621670e 2023-11-27 14:54:50,273 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103184af19c8d794edb8e84878ce082cb48 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103184af19c8d794edb8e84878ce082cb48 2023-11-27 14:54:50,273 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-11-27 14:54:50,273 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-27 14:54:50,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:50,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:50,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:50,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:50,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:50,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:50,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:50,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:50,317 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): b70da5c01d91e889a747889289d215f3#family1#compaction#33 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:54:50,739 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/5fac37adc31a43d9a97d47dc20a043c0 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5fac37adc31a43d9a97d47dc20a043c0 2023-11-27 14:54:50,745 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in b70da5c01d91e889a747889289d215f3/family1 of b70da5c01d91e889a747889289d215f3 into 5fac37adc31a43d9a97d47dc20a043c0(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:54:50,745 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:50,746 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1701096890261; duration=0sec 2023-11-27 14:54:50,746 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:50,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/7b6782a3818044af98d7617198b9c7e4, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c3d859c50b5b42608ae34600f3535ce1, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/36f0b1c75aac419bad50b06507ccfc9b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/0d72b1c0e9b24d21a7c77fc174049011, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/30b1dbed3d7a4b4b8e93aa7b794dc887, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/aafbe6433db344b793e02c4c8ffd8df0, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5846e05a6b41463bbd8815264fe57a05, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/ae94e7f56bfd47138b40370b8d65cc48, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/2fd9d5d192d44c6790eef7f41318ad02, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c37510aeb8ba4af1b7a7e55151f036f5_SeqId_44_] to archive 2023-11-27 14:54:50,763 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:50,765 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/7b6782a3818044af98d7617198b9c7e4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/7b6782a3818044af98d7617198b9c7e4 2023-11-27 14:54:50,767 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c3d859c50b5b42608ae34600f3535ce1 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c3d859c50b5b42608ae34600f3535ce1 2023-11-27 14:54:50,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/36f0b1c75aac419bad50b06507ccfc9b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/36f0b1c75aac419bad50b06507ccfc9b 2023-11-27 14:54:50,770 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/0d72b1c0e9b24d21a7c77fc174049011 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/0d72b1c0e9b24d21a7c77fc174049011 2023-11-27 14:54:50,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/30b1dbed3d7a4b4b8e93aa7b794dc887 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/30b1dbed3d7a4b4b8e93aa7b794dc887 2023-11-27 14:54:50,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/aafbe6433db344b793e02c4c8ffd8df0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/aafbe6433db344b793e02c4c8ffd8df0 2023-11-27 14:54:50,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5846e05a6b41463bbd8815264fe57a05 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5846e05a6b41463bbd8815264fe57a05 2023-11-27 14:54:50,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/ae94e7f56bfd47138b40370b8d65cc48 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/ae94e7f56bfd47138b40370b8d65cc48 2023-11-27 14:54:50,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/2fd9d5d192d44c6790eef7f41318ad02 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/2fd9d5d192d44c6790eef7f41318ad02 2023-11-27 14:54:50,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c37510aeb8ba4af1b7a7e55151f036f5_SeqId_44_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/c37510aeb8ba4af1b7a7e55151f036f5_SeqId_44_ 2023-11-27 14:54:51,139 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205103b0dcbf36d41d08a13b8cd86bce795 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205103b0dcbf36d41d08a13b8cd86bce795 2023-11-27 14:54:51,140 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:54:51,171 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e52ca2bb45c94a9584367b54cadabe86 first=Optional[r1] last=Optional[r2] 2023-11-27 14:54:51,172 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e52ca2bb45c94a9584367b54cadabe86}] 2023-11-27 14:54:51,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e52ca2bb45c94a9584367b54cadabe86 for inclusion in b70da5c01d91e889a747889289d215f3/family1 2023-11-27 14:54:51,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-11-27 14:54:51,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:54:51,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:51,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e52ca2bb45c94a9584367b54cadabe86 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__l79s2hjj3fbq7f3kpv3ugm7t8mmc93vd3frln0o88erkbeee9mmrh1rcojfst13n/family1/e52ca2bb45c94a9584367b54cadabe86 2023-11-27 14:54:51,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__l79s2hjj3fbq7f3kpv3ugm7t8mmc93vd3frln0o88erkbeee9mmrh1rcojfst13n/family1/e52ca2bb45c94a9584367b54cadabe86 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/9b87cd04f6dc427f9693fe39cc0e74b2_SeqId_47_ 2023-11-27 14:54:51,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__l79s2hjj3fbq7f3kpv3ugm7t8mmc93vd3frln0o88erkbeee9mmrh1rcojfst13n/family1/e52ca2bb45c94a9584367b54cadabe86 into b70da5c01d91e889a747889289d215f3/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/9b87cd04f6dc427f9693fe39cc0e74b2_SeqId_47_ - updating store file list. 2023-11-27 14:54:51,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/9b87cd04f6dc427f9693fe39cc0e74b2_SeqId_47_ into b70da5c01d91e889a747889289d215f3/family1 2023-11-27 14:54:51,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__l79s2hjj3fbq7f3kpv3ugm7t8mmc93vd3frln0o88erkbeee9mmrh1rcojfst13n/family1/e52ca2bb45c94a9584367b54cadabe86 into b70da5c01d91e889a747889289d215f3/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/9b87cd04f6dc427f9693fe39cc0e74b2_SeqId_47_) 2023-11-27 14:54:51,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__l79s2hjj3fbq7f3kpv3ugm7t8mmc93vd3frln0o88erkbeee9mmrh1rcojfst13n/family1/e52ca2bb45c94a9584367b54cadabe86 2023-11-27 14:54:51,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-27 14:54:51,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-27 14:54:51,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): b70da5c01d91e889a747889289d215f3/family1 is initiating minor compaction (all files) 2023-11-27 14:54:51,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@8b61d25; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:51,192 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b70da5c01d91e889a747889289d215f3/family1 in testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:54:51,192 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/bafa20afd048478292c655ae3dda41f4, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5fac37adc31a43d9a97d47dc20a043c0, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/9b87cd04f6dc427f9693fe39cc0e74b2_SeqId_47_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp, totalSize=176.8 K 2023-11-27 14:54:51,192 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting bafa20afd048478292c655ae3dda41f4, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878813266 2023-11-27 14:54:51,193 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 5fac37adc31a43d9a97d47dc20a043c0, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621213266 2023-11-27 14:54:51,194 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 9b87cd04f6dc427f9693fe39cc0e74b2_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965213266 2023-11-27 14:54:51,194 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:54:51,195 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:51,197 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512012bef2b7045684945a41eae549cefd967 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512012bef2b7045684945a41eae549cefd967 2023-11-27 14:54:51,198 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120567691713b3fa4ad296fac88025bbdeb3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120567691713b3fa4ad296fac88025bbdeb3 2023-11-27 14:54:51,198 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-11-27 14:54:51,198 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-27 14:54:51,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:54:51,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:54:51,199 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:51,200 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:54:51,202 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8590 sec 2023-11-27 14:54:51,215 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): b70da5c01d91e889a747889289d215f3#family1#compaction#34 average throughput is 14.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:54:51,245 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/.tmp/family1/20f78a42821648d4b03a7e866bd72db0 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/20f78a42821648d4b03a7e866bd72db0 2023-11-27 14:54:51,253 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b70da5c01d91e889a747889289d215f3/family1 of b70da5c01d91e889a747889289d215f3 into 20f78a42821648d4b03a7e866bd72db0(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:54:51,253 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:54:51,253 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1701096891192; duration=0sec 2023-11-27 14:54:51,253 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:51,316 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=258 (was 267), OpenFileDescriptor=647 (was 658), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=293 (was 320), ProcessCount=168 (was 168), AvailableMemoryMB=8313 (was 8413) 2023-11-27 14:54:51,327 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=258, OpenFileDescriptor=647, MaxFileDescriptor=60000, SystemLoadAverage=293, ProcessCount=168, AvailableMemoryMB=8312 2023-11-27 14:54:51,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-27 14:54:51,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:51,332 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:54:51,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-11-27 14:54:51,333 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:54:51,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-27 14:54:51,335 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:51,335 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:51,335 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:51,336 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6 empty. 2023-11-27 14:54:51,336 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815 empty. 2023-11-27 14:54:51,336 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277 empty. 2023-11-27 14:54:51,336 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:51,336 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:51,336 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:51,337 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-11-27 14:54:51,344 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-11-27 14:54:51,346 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => e13a129f6f563cfb251e64b3f5d12815, NAME => 'testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:54:51,346 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 4e630d9c8bdef52b2f9df3adf2c55277, NAME => 'testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:54:51,346 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 56c85fb3659c83b26d25644563d1ffc6, NAME => 'testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:54:51,360 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/bafa20afd048478292c655ae3dda41f4, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5fac37adc31a43d9a97d47dc20a043c0, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/9b87cd04f6dc427f9693fe39cc0e74b2_SeqId_47_] to archive 2023-11-27 14:54:51,361 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:51,362 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/bafa20afd048478292c655ae3dda41f4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/bafa20afd048478292c655ae3dda41f4 2023-11-27 14:54:51,364 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5fac37adc31a43d9a97d47dc20a043c0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/5fac37adc31a43d9a97d47dc20a043c0 2023-11-27 14:54:51,365 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/9b87cd04f6dc427f9693fe39cc0e74b2_SeqId_47_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/family1/9b87cd04f6dc427f9693fe39cc0e74b2_SeqId_47_ 2023-11-27 14:54:51,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-27 14:54:51,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-27 14:54:51,773 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:51,773 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:51,773 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 4e630d9c8bdef52b2f9df3adf2c55277, disabling compactions & flushes 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing e13a129f6f563cfb251e64b3f5d12815, disabling compactions & flushes 2023-11-27 14:54:51,774 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. after waiting 0 ms 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:51,774 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:51,774 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 4e630d9c8bdef52b2f9df3adf2c55277: 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. after waiting 0 ms 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 56c85fb3659c83b26d25644563d1ffc6, disabling compactions & flushes 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:51,774 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:51,774 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. after waiting 0 ms 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:51,775 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:51,775 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 56c85fb3659c83b26d25644563d1ffc6: 2023-11-27 14:54:51,774 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for e13a129f6f563cfb251e64b3f5d12815: 2023-11-27 14:54:51,777 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:54:51,778 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1701096891778"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096891778"}]},"ts":"1701096891778"} 2023-11-27 14:54:51,778 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701096891778"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096891778"}]},"ts":"1701096891778"} 2023-11-27 14:54:51,778 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701096891778"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096891778"}]},"ts":"1701096891778"} 2023-11-27 14:54:51,781 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-27 14:54:51,782 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:54:51,782 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096891782"}]},"ts":"1701096891782"} 2023-11-27 14:54:51,784 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-11-27 14:54:51,788 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=56c85fb3659c83b26d25644563d1ffc6, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4e630d9c8bdef52b2f9df3adf2c55277, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=e13a129f6f563cfb251e64b3f5d12815, ASSIGN}] 2023-11-27 14:54:51,789 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4e630d9c8bdef52b2f9df3adf2c55277, ASSIGN 2023-11-27 14:54:51,790 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=56c85fb3659c83b26d25644563d1ffc6, ASSIGN 2023-11-27 14:54:51,790 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=e13a129f6f563cfb251e64b3f5d12815, ASSIGN 2023-11-27 14:54:51,790 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4e630d9c8bdef52b2f9df3adf2c55277, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:54:51,792 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=56c85fb3659c83b26d25644563d1ffc6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:54:51,792 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=e13a129f6f563cfb251e64b3f5d12815, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:54:51,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-27 14:54:51,942 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4e630d9c8bdef52b2f9df3adf2c55277, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:51,942 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=56c85fb3659c83b26d25644563d1ffc6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:51,942 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1701096891942"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096891942"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096891942"}]},"ts":"1701096891942"} 2023-11-27 14:54:51,942 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=e13a129f6f563cfb251e64b3f5d12815, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:51,942 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701096891942"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096891942"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096891942"}]},"ts":"1701096891942"} 2023-11-27 14:54:51,942 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701096891942"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096891942"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096891942"}]},"ts":"1701096891942"} 2023-11-27 14:54:51,944 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=31, ppid=29, state=RUNNABLE; OpenRegionProcedure 4e630d9c8bdef52b2f9df3adf2c55277, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:51,945 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure 56c85fb3659c83b26d25644563d1ffc6, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:51,946 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure e13a129f6f563cfb251e64b3f5d12815, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:52,097 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:52,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4e630d9c8bdef52b2f9df3adf2c55277, NAME => 'testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277.', STARTKEY => '1', ENDKEY => '2'} 2023-11-27 14:54:52,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:52,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:52,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:52,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:52,102 INFO [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:52,104 DEBUG [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277/family1 2023-11-27 14:54:52,104 DEBUG [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277/family1 2023-11-27 14:54:52,104 INFO [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4e630d9c8bdef52b2f9df3adf2c55277 columnFamilyName family1 2023-11-27 14:54:52,106 INFO [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] regionserver.HStore(310): Store=4e630d9c8bdef52b2f9df3adf2c55277/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:52,106 INFO [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:52,107 DEBUG [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277/family2 2023-11-27 14:54:52,107 DEBUG [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277/family2 2023-11-27 14:54:52,108 INFO [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4e630d9c8bdef52b2f9df3adf2c55277 columnFamilyName family2 2023-11-27 14:54:52,109 INFO [StoreOpener-4e630d9c8bdef52b2f9df3adf2c55277-1] regionserver.HStore(310): Store=4e630d9c8bdef52b2f9df3adf2c55277/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:52,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:52,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:52,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:54:52,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:54:52,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:52,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4e630d9c8bdef52b2f9df3adf2c55277; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62236170, jitterRate=-0.07260879874229431}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:54:52,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4e630d9c8bdef52b2f9df3adf2c55277: 2023-11-27 14:54:52,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277., pid=31, masterSystemTime=1701096892097 2023-11-27 14:54:52,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:52,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:52,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:52,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e13a129f6f563cfb251e64b3f5d12815, NAME => 'testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815.', STARTKEY => '2', ENDKEY => ''} 2023-11-27 14:54:52,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:52,119 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=4e630d9c8bdef52b2f9df3adf2c55277, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:52,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:52,119 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1701096892119"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096892119"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096892119"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096892119"}]},"ts":"1701096892119"} 2023-11-27 14:54:52,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:52,121 INFO [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:52,122 DEBUG [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1 2023-11-27 14:54:52,122 DEBUG [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1 2023-11-27 14:54:52,123 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=31, resume processing ppid=29 2023-11-27 14:54:52,123 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=31, ppid=29, state=SUCCESS; OpenRegionProcedure 4e630d9c8bdef52b2f9df3adf2c55277, server=jenkins-hbase4.apache.org,42669,1701096850360 in 177 msec 2023-11-27 14:54:52,123 INFO [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e13a129f6f563cfb251e64b3f5d12815 columnFamilyName family1 2023-11-27 14:54:52,124 INFO [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] regionserver.HStore(310): Store=e13a129f6f563cfb251e64b3f5d12815/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:52,124 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4e630d9c8bdef52b2f9df3adf2c55277, ASSIGN in 335 msec 2023-11-27 14:54:52,124 INFO [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:52,126 DEBUG [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family2 2023-11-27 14:54:52,126 DEBUG [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family2 2023-11-27 14:54:52,127 INFO [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e13a129f6f563cfb251e64b3f5d12815 columnFamilyName family2 2023-11-27 14:54:52,128 INFO [StoreOpener-e13a129f6f563cfb251e64b3f5d12815-1] regionserver.HStore(310): Store=e13a129f6f563cfb251e64b3f5d12815/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:52,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:52,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:52,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:54:52,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:54:52,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:52,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e13a129f6f563cfb251e64b3f5d12815; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67273890, jitterRate=0.0024590790271759033}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:54:52,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e13a129f6f563cfb251e64b3f5d12815: 2023-11-27 14:54:52,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815., pid=33, masterSystemTime=1701096892097 2023-11-27 14:54:52,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:52,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:52,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:52,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 56c85fb3659c83b26d25644563d1ffc6, NAME => 'testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6.', STARTKEY => '', ENDKEY => '1'} 2023-11-27 14:54:52,136 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=e13a129f6f563cfb251e64b3f5d12815, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:52,137 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701096892136"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096892136"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096892136"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096892136"}]},"ts":"1701096892136"} 2023-11-27 14:54:52,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:52,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:52,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:52,138 INFO [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:52,140 DEBUG [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6/family1 2023-11-27 14:54:52,140 DEBUG [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6/family1 2023-11-27 14:54:52,140 INFO [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 56c85fb3659c83b26d25644563d1ffc6 columnFamilyName family1 2023-11-27 14:54:52,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=33, resume processing ppid=30 2023-11-27 14:54:52,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure e13a129f6f563cfb251e64b3f5d12815, server=jenkins-hbase4.apache.org,42669,1701096850360 in 193 msec 2023-11-27 14:54:52,141 INFO [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] regionserver.HStore(310): Store=56c85fb3659c83b26d25644563d1ffc6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:52,141 INFO [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:52,141 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=e13a129f6f563cfb251e64b3f5d12815, ASSIGN in 352 msec 2023-11-27 14:54:52,143 DEBUG [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6/family2 2023-11-27 14:54:52,143 DEBUG [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6/family2 2023-11-27 14:54:52,143 INFO [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 56c85fb3659c83b26d25644563d1ffc6 columnFamilyName family2 2023-11-27 14:54:52,144 INFO [StoreOpener-56c85fb3659c83b26d25644563d1ffc6-1] regionserver.HStore(310): Store=56c85fb3659c83b26d25644563d1ffc6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:52,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:52,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:52,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:54:52,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:54:52,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:52,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 56c85fb3659c83b26d25644563d1ffc6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71312955, jitterRate=0.06264583766460419}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:54:52,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 56c85fb3659c83b26d25644563d1ffc6: 2023-11-27 14:54:52,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6., pid=32, masterSystemTime=1701096892097 2023-11-27 14:54:52,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:52,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:52,153 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=56c85fb3659c83b26d25644563d1ffc6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,153 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1701096892153"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096892153"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096892153"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096892153"}]},"ts":"1701096892153"} 2023-11-27 14:54:52,156 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=32, resume processing ppid=28 2023-11-27 14:54:52,156 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure 56c85fb3659c83b26d25644563d1ffc6, server=jenkins-hbase4.apache.org,42669,1701096850360 in 209 msec 2023-11-27 14:54:52,158 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=28, resume processing ppid=27 2023-11-27 14:54:52,158 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=56c85fb3659c83b26d25644563d1ffc6, ASSIGN in 368 msec 2023-11-27 14:54:52,159 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:54:52,159 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096892159"}]},"ts":"1701096892159"} 2023-11-27 14:54:52,160 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-11-27 14:54:52,162 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:54:52,164 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 832 msec 2023-11-27 14:54:52,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-27 14:54:52,438 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-11-27 14:54:52,445 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:52,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,448 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-11-27 14:54:52,448 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:52,448 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-11-27 14:54:52,448 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-11-27 14:54:52,448 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,448 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,449 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,449 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:52,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:52,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:52,450 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,450 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:52,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:52,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,451 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,451 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,451 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-11-27 14:54:52,451 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:52,451 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-11-27 14:54:52,451 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:52,451 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. started... 2023-11-27 14:54:52,451 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:52,451 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:54:52,452 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:52,452 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:52,452 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4e630d9c8bdef52b2f9df3adf2c55277: 2023-11-27 14:54:52,451 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. started... 2023-11-27 14:54:52,452 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:52,452 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. started... 2023-11-27 14:54:52,453 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing e13a129f6f563cfb251e64b3f5d12815 2/2 column families, dataSize=48 B heapSize=632 B 2023-11-27 14:54:52,454 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 56c85fb3659c83b26d25644563d1ffc6: 2023-11-27 14:54:52,454 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:52,452 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:54:52,462 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:54:52,475 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127334a2ca7737d49c0bfe9413509818f36 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231127334a2ca7737d49c0bfe9413509818f36 2023-11-27 14:54:52,881 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/.tmp/family1/5e59bc3e17ad413a89770bbdacffa793 2023-11-27 14:54:52,887 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/.tmp/family1/5e59bc3e17ad413a89770bbdacffa793 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/5e59bc3e17ad413a89770bbdacffa793 2023-11-27 14:54:52,892 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/5e59bc3e17ad413a89770bbdacffa793, entries=1, sequenceid=5, filesize=4.9 K 2023-11-27 14:54:52,893 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for e13a129f6f563cfb251e64b3f5d12815 in 440ms, sequenceid=5, compaction requested=false 2023-11-27 14:54:52,893 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-11-27 14:54:52,894 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for e13a129f6f563cfb251e64b3f5d12815: 2023-11-27 14:54:52,894 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:52,894 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:54:52,894 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:54:52,894 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,894 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-11-27 14:54:52,894 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-11-27 14:54:52,897 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,897 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:52,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:52,898 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,898 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:52,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:52,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:52,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:52,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-11-27 14:54:52,899 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-11-27 14:54:52,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52f42f3b[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:52,899 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,901 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,901 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,901 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:52,901 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-11-27 14:54:52,901 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-11-27 14:54:52,901 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:52,903 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,903 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-11-27 14:54:52,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:52,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:52,903 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:52,903 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-11-27 14:54:52,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:52,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:52,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:52,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:52,905 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-11-27 14:54:52,906 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-11-27 14:54:52,906 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:52,906 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:52,906 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,906 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:52,907 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,907 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:52,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:52,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,907 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:52,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:52,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:52,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:52,908 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:52,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,910 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:52,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,913 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,913 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:52,913 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:52,913 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:52,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-11-27 14:54:52,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:52,913 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:52,913 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:52,914 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:52,914 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,914 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,914 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:52,914 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:52,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:52,914 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:52,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:53,014 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:53,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:53,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:53,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:53,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,023 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-11-27 14:54:53,023 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:53,023 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-11-27 14:54:53,023 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-11-27 14:54:53,023 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,023 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,025 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:53,025 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:53,025 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,025 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:53,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,026 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,027 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-11-27 14:54:53,027 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,027 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-11-27 14:54:53,027 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:53,027 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-11-27 14:54:53,027 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:53,027 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. started... 2023-11-27 14:54:53,027 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:53,027 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. started... 2023-11-27 14:54:53,027 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4e630d9c8bdef52b2f9df3adf2c55277: 2023-11-27 14:54:53,027 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:54:53,027 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing e13a129f6f563cfb251e64b3f5d12815 2/2 column families, dataSize=90 B heapSize=752 B 2023-11-27 14:54:53,028 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:54:53,028 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:53,028 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. started... 2023-11-27 14:54:53,028 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:53,028 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 56c85fb3659c83b26d25644563d1ffc6: 2023-11-27 14:54:53,028 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:54:53,028 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:54:53,028 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:54:53,444 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311278f823d5d8aa743fea31d860fb23a8298 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202311278f823d5d8aa743fea31d860fb23a8298 2023-11-27 14:54:53,850 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/.tmp/family1/8ed04771413041f89c255c1fb6cab080 2023-11-27 14:54:53,855 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/.tmp/family1/8ed04771413041f89c255c1fb6cab080 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/8ed04771413041f89c255c1fb6cab080 2023-11-27 14:54:53,860 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/8ed04771413041f89c255c1fb6cab080, entries=2, sequenceid=9, filesize=5.0 K 2023-11-27 14:54:53,861 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for e13a129f6f563cfb251e64b3f5d12815 in 834ms, sequenceid=9, compaction requested=false 2023-11-27 14:54:53,862 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for e13a129f6f563cfb251e64b3f5d12815: 2023-11-27 14:54:53,862 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:54:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:54:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-11-27 14:54:53,862 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-11-27 14:54:53,864 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,864 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:53,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:53,864 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,864 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:53,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:53,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:53,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:53,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-11-27 14:54:53,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@401711d6[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:53,866 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-11-27 14:54:53,867 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,869 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,869 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-11-27 14:54:53,869 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-11-27 14:54:53,869 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:53,869 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,870 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:53,871 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,871 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-11-27 14:54:53,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:53,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:53,871 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:53,871 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-11-27 14:54:53,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:53,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:53,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:53,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:53,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-11-27 14:54:53,874 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-11-27 14:54:53,874 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:53,874 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:53,874 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,874 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:53,875 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,875 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:53,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:53,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,876 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:53,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:53,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:53,876 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:53,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:53,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,878 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:53,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,884 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,884 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,885 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,885 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:53,885 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:53,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:53,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-11-27 14:54:53,885 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:53,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:53,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-11-27 14:54:53,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:53,885 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:53,885 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,885 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,885 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,885 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:53,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:53,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:53,886 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:53,986 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:53,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:53,994 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-11-27 14:54:53,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-27 14:54:53,996 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-27 14:54:53,997 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097493997 2023-11-27 14:54:53,997 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:53,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:54:53,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-11-27 14:54:53,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-27 14:54:54,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x381c1de3 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:54:54,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7a783065, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:54:54,006 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231127 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-11-27 14:54:54,058 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:54:54,058 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3e655241 to 127.0.0.1:61098 2023-11-27 14:54:54,058 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:54:54,060 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:54:54,061 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0073972b to 127.0.0.1:61098 2023-11-27 14:54:54,061 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:54:54,061 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:54:54,062 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x36434290 to 127.0.0.1:61098 2023-11-27 14:54:54,062 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:54:54,062 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:54:54,062 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x587a02ee to 127.0.0.1:61098 2023-11-27 14:54:54,062 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:54:54,872 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127166dd4985d08438e89cd1e57919530d4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231127166dd4985d08438e89cd1e57919530d4 2023-11-27 14:54:54,873 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-27 14:54:54,887 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:54:54,888 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38882, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:54:54,896 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:54:54,898 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47364, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:54:54,907 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231127/family1/ca590e407ee44abab11e5c48fc08adb1 first=Optional[k0] last=Optional[k1] 2023-11-27 14:54:54,907 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231127/family1/ca590e407ee44abab11e5c48fc08adb1}] 2023-11-27 14:54:54,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231127/family1/ca590e407ee44abab11e5c48fc08adb1 for inclusion in e13a129f6f563cfb251e64b3f5d12815/family1 2023-11-27 14:54:54,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-11-27 14:54:54,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-27 14:54:54,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for e13a129f6f563cfb251e64b3f5d12815: 2023-11-27 14:54:54,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231127/family1/ca590e407ee44abab11e5c48fc08adb1 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testScannerOnBulkLoadRefHFiles__eita4e6jpf2d3qulaeap66nhsvm9tmp2v7tbhoet3rbk11i2rd5rvv1s7koukhch/family1/ca590e407ee44abab11e5c48fc08adb1 2023-11-27 14:54:54,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testScannerOnBulkLoadRefHFiles__eita4e6jpf2d3qulaeap66nhsvm9tmp2v7tbhoet3rbk11i2rd5rvv1s7koukhch/family1/ca590e407ee44abab11e5c48fc08adb1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/2eca5f752cf447029f61c6336ed8418f_SeqId_12_ 2023-11-27 14:54:54,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testScannerOnBulkLoadRefHFiles__eita4e6jpf2d3qulaeap66nhsvm9tmp2v7tbhoet3rbk11i2rd5rvv1s7koukhch/family1/ca590e407ee44abab11e5c48fc08adb1 into e13a129f6f563cfb251e64b3f5d12815/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/2eca5f752cf447029f61c6336ed8418f_SeqId_12_ - updating store file list. 2023-11-27 14:54:54,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/2eca5f752cf447029f61c6336ed8418f_SeqId_12_ into e13a129f6f563cfb251e64b3f5d12815/family1 2023-11-27 14:54:54,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testScannerOnBulkLoadRefHFiles__eita4e6jpf2d3qulaeap66nhsvm9tmp2v7tbhoet3rbk11i2rd5rvv1s7koukhch/family1/ca590e407ee44abab11e5c48fc08adb1 into e13a129f6f563cfb251e64b3f5d12815/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/2eca5f752cf447029f61c6336ed8418f_SeqId_12_) 2023-11-27 14:54:54,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testScannerOnBulkLoadRefHFiles__eita4e6jpf2d3qulaeap66nhsvm9tmp2v7tbhoet3rbk11i2rd5rvv1s7koukhch/family1/ca590e407ee44abab11e5c48fc08adb1 2023-11-27 14:54:54,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-27 14:54:54,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-27 14:54:54,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): e13a129f6f563cfb251e64b3f5d12815/family1 is initiating minor compaction (all files) 2023-11-27 14:54:54,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@77340efd; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:54,925 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e13a129f6f563cfb251e64b3f5d12815/family1 in testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:54:54,925 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/5e59bc3e17ad413a89770bbdacffa793, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/8ed04771413041f89c255c1fb6cab080, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/2eca5f752cf447029f61c6336ed8418f_SeqId_12_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/.tmp, totalSize=15.1 K 2023-11-27 14:54:54,925 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 5e59bc3e17ad413a89770bbdacffa793, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096892438 2023-11-27 14:54:54,926 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 8ed04771413041f89c255c1fb6cab080, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096892438 2023-11-27 14:54:54,927 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 2eca5f752cf447029f61c6336ed8418f_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096892438 2023-11-27 14:54:54,927 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-27 14:54:54,928 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:54,930 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231127334a2ca7737d49c0bfe9413509818f36 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231127334a2ca7737d49c0bfe9413509818f36 2023-11-27 14:54:54,931 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202311278f823d5d8aa743fea31d860fb23a8298 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202311278f823d5d8aa743fea31d860fb23a8298 2023-11-27 14:54:54,931 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-11-27 14:54:54,931 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231127' 2023-11-27 14:54:54,931 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231127 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-11-27 14:54:54,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-27 14:54:54,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:54:54,933 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-27 14:54:54,934 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 938 msec 2023-11-27 14:54:54,938 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): e13a129f6f563cfb251e64b3f5d12815#family1#compaction#37 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:54:54,977 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=257 (was 258), OpenFileDescriptor=622 (was 647), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=310 (was 293) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=8251 (was 8312) 2023-11-27 14:54:54,992 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=257, OpenFileDescriptor=622, MaxFileDescriptor=60000, SystemLoadAverage=310, ProcessCount=168, AvailableMemoryMB=8250 2023-11-27 14:54:54,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-27 14:54:54,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:54,996 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:54:54,996 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-11-27 14:54:54,997 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:54:54,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-27 14:54:54,998 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba 2023-11-27 14:54:54,998 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba empty. 2023-11-27 14:54:54,999 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba 2023-11-27 14:54:54,999 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-11-27 14:54:55,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-27 14:54:55,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-27 14:54:55,413 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-27 14:54:55,415 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8ae027c1211c228b679773750f57d6ba, NAME => 'testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:54:55,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-27 14:54:55,759 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/.tmp/family1/a883fe28d9d44f6e946ef6ce169a9545 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/a883fe28d9d44f6e946ef6ce169a9545 2023-11-27 14:54:55,764 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in e13a129f6f563cfb251e64b3f5d12815/family1 of e13a129f6f563cfb251e64b3f5d12815 into a883fe28d9d44f6e946ef6ce169a9545(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:54:55,765 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e13a129f6f563cfb251e64b3f5d12815: 2023-11-27 14:54:55,765 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1701096894924; duration=0sec 2023-11-27 14:54:55,765 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:54:55,821 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:55,821 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 8ae027c1211c228b679773750f57d6ba, disabling compactions & flushes 2023-11-27 14:54:55,821 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:55,821 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:55,821 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. after waiting 0 ms 2023-11-27 14:54:55,821 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:55,821 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:55,821 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:54:55,824 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:54:55,825 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701096895825"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096895825"}]},"ts":"1701096895825"} 2023-11-27 14:54:55,826 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-27 14:54:55,827 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:54:55,827 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096895827"}]},"ts":"1701096895827"} 2023-11-27 14:54:55,828 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-11-27 14:54:55,831 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8ae027c1211c228b679773750f57d6ba, ASSIGN}] 2023-11-27 14:54:55,833 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8ae027c1211c228b679773750f57d6ba, ASSIGN 2023-11-27 14:54:55,833 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8ae027c1211c228b679773750f57d6ba, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:54:55,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/5e59bc3e17ad413a89770bbdacffa793, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/8ed04771413041f89c255c1fb6cab080, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/2eca5f752cf447029f61c6336ed8418f_SeqId_12_] to archive 2023-11-27 14:54:55,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:54:55,857 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/5e59bc3e17ad413a89770bbdacffa793 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/5e59bc3e17ad413a89770bbdacffa793 2023-11-27 14:54:55,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/8ed04771413041f89c255c1fb6cab080 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/8ed04771413041f89c255c1fb6cab080 2023-11-27 14:54:55,860 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/2eca5f752cf447029f61c6336ed8418f_SeqId_12_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/family1/2eca5f752cf447029f61c6336ed8418f_SeqId_12_ 2023-11-27 14:54:55,984 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=8ae027c1211c228b679773750f57d6ba, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:55,985 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701096895984"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096895984"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096895984"}]},"ts":"1701096895984"} 2023-11-27 14:54:55,986 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 8ae027c1211c228b679773750f57d6ba, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:54:56,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-27 14:54:56,138 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:56,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:56,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8ae027c1211c228b679773750f57d6ba, NAME => 'testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:54:56,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 8ae027c1211c228b679773750f57d6ba 2023-11-27 14:54:56,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:54:56,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8ae027c1211c228b679773750f57d6ba 2023-11-27 14:54:56,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8ae027c1211c228b679773750f57d6ba 2023-11-27 14:54:56,143 INFO [StoreOpener-8ae027c1211c228b679773750f57d6ba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8ae027c1211c228b679773750f57d6ba 2023-11-27 14:54:56,144 DEBUG [StoreOpener-8ae027c1211c228b679773750f57d6ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1 2023-11-27 14:54:56,144 DEBUG [StoreOpener-8ae027c1211c228b679773750f57d6ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1 2023-11-27 14:54:56,145 INFO [StoreOpener-8ae027c1211c228b679773750f57d6ba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ae027c1211c228b679773750f57d6ba columnFamilyName family1 2023-11-27 14:54:56,146 INFO [StoreOpener-8ae027c1211c228b679773750f57d6ba-1] regionserver.HStore(310): Store=8ae027c1211c228b679773750f57d6ba/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:54:56,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba 2023-11-27 14:54:56,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba 2023-11-27 14:54:56,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8ae027c1211c228b679773750f57d6ba 2023-11-27 14:54:56,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:54:56,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8ae027c1211c228b679773750f57d6ba; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75174598, jitterRate=0.12018880248069763}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:54:56,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:54:56,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba., pid=37, masterSystemTime=1701096896138 2023-11-27 14:54:56,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:56,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:56,156 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=8ae027c1211c228b679773750f57d6ba, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:56,156 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1701096896156"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096896156"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096896156"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096896156"}]},"ts":"1701096896156"} 2023-11-27 14:54:56,159 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=37, resume processing ppid=36 2023-11-27 14:54:56,159 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 8ae027c1211c228b679773750f57d6ba, server=jenkins-hbase4.apache.org,42669,1701096850360 in 171 msec 2023-11-27 14:54:56,161 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=36, resume processing ppid=35 2023-11-27 14:54:56,161 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8ae027c1211c228b679773750f57d6ba, ASSIGN in 328 msec 2023-11-27 14:54:56,161 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:54:56,161 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096896161"}]},"ts":"1701096896161"} 2023-11-27 14:54:56,162 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-11-27 14:54:56,166 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:54:56,167 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1720 sec 2023-11-27 14:54:57,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-27 14:54:57,103 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-11-27 14:54:57,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:57,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,134 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:54:57,134 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:57,134 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:57,134 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:57,135 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,135 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,136 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:57,136 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:57,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:57,137 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,137 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:57,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:57,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,138 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:57,138 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:57,138 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:57,140 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:57,140 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:57,140 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:57,140 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:54:57,141 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-27 14:54:57,565 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511301358e2c3e19f4144af39b4b3c2d2c47b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511301358e2c3e19f4144af39b4b3c2d2c47b 2023-11-27 14:54:57,643 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:54:57,969 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/25b99bb6af194a66b462358d7cad85f1 2023-11-27 14:54:57,974 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/25b99bb6af194a66b462358d7cad85f1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/25b99bb6af194a66b462358d7cad85f1 2023-11-27 14:54:57,979 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/25b99bb6af194a66b462358d7cad85f1, entries=1000, sequenceid=5, filesize=165.1 K 2023-11-27 14:54:57,980 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 839ms, sequenceid=5, compaction requested=false 2023-11-27 14:54:57,980 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:54:57,981 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:54:57,981 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:57,981 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:57,981 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:57,981 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,981 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:57,981 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:57,983 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,983 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:57,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:57,983 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,983 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:57,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:57,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:57,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:57,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:57,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f323e1f[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:57,985 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:57,985 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,986 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,987 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:57,987 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:57,987 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:57,987 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,987 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:57,993 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:57,993 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:57,993 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,993 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:57,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:57,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:57,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:57,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:57,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:57,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:57,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:57,995 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:57,995 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:57,995 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:57,995 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,995 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:57,997 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,997 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:57,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:57,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,997 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:57,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:57,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:57,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:57,997 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:57,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:57,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,999 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:57,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:57,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:58,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:58,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:54:58,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,002 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:58,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:58,002 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:58,102 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:58,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:58,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:58,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:58,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,110 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:54:58,110 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:58,110 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:58,110 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:58,111 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,111 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,112 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,112 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:58,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:58,112 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,112 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:58,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:58,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,114 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:58,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:58,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:58,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:58,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:58,115 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:58,115 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:58,115 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:54:58,115 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-27 14:54:58,528 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112830c11578367f496aad6d3801274cd56d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112830c11578367f496aad6d3801274cd56d 2023-11-27 14:54:58,533 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/7fd24fbf1fe24de1ace1788e470fc3fa 2023-11-27 14:54:58,538 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/7fd24fbf1fe24de1ace1788e470fc3fa as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7fd24fbf1fe24de1ace1788e470fc3fa 2023-11-27 14:54:58,542 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7fd24fbf1fe24de1ace1788e470fc3fa, entries=1, sequenceid=9, filesize=5.0 K 2023-11-27 14:54:58,542 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 427ms, sequenceid=9, compaction requested=false 2023-11-27 14:54:58,543 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:54:58,543 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:58,543 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:58,543 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:58,543 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,543 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:58,543 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:58,546 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,546 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:58,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:58,546 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,546 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:58,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:58,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:58,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:58,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:58,548 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14579882[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:58,548 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:58,548 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,549 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,549 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:58,550 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:58,550 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:58,550 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,550 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:58,551 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:58,551 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,551 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:58,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:58,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:58,551 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:58,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:58,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:58,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:58,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:58,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:58,555 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:58,555 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:58,555 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:58,555 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,555 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:58,556 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,556 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,556 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:58,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:58,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:58,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:58,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:58,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:58,557 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:58,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,558 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:58,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:58,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:54:58,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:58,566 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,566 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:58,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:58,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:58,667 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:58,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:58,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:58,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:58,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,673 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:54:58,674 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:58,674 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:58,674 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:58,674 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,674 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,675 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,675 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:58,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:58,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:58,676 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:58,676 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:58,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,676 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:58,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,677 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,677 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:58,677 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:58,677 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:58,677 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:58,677 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:58,677 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:58,677 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:58,678 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:58,678 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:54:58,678 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:59,091 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120103313e14a6664c38816cd5dffe7940f8 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120103313e14a6664c38816cd5dffe7940f8 2023-11-27 14:54:59,498 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/ab323b443aa24eabb8b2406b93474d07 2023-11-27 14:54:59,503 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/ab323b443aa24eabb8b2406b93474d07 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/ab323b443aa24eabb8b2406b93474d07 2023-11-27 14:54:59,507 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/ab323b443aa24eabb8b2406b93474d07, entries=1, sequenceid=13, filesize=4.9 K 2023-11-27 14:54:59,507 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 829ms, sequenceid=13, compaction requested=true 2023-11-27 14:54:59,507 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:54:59,508 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:59,508 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:54:59,508 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:54:59,508 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,508 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:54:59,508 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:54:59,509 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,509 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:59,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:59,510 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,510 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:54:59,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:59,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:59,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:59,511 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:54:59,511 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67a9e0c4[Count = 0] remaining members to acquire global barrier 2023-11-27 14:54:59,511 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:54:59,511 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,513 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,513 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:54:59,513 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:54:59,513 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:54:59,513 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,513 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:54:59,515 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,515 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:54:59,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:59,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:59,515 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:59,516 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:54:59,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:59,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:59,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:59,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:54:59,518 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:54:59,518 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:54:59,518 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:54:59,518 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:54:59,518 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,518 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:54:59,520 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,520 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,520 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:59,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:54:59,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:54:59,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:54:59,520 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:59,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:59,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:54:59,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,522 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:54:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,524 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,524 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,524 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:59,524 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:59,524 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,524 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:54:59,524 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:59,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:54:59,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:54:59,524 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:54:59,524 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,525 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:54:59,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:54:59,525 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:54:59,525 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:54:59,625 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:54:59,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:54:59,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:54:59,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:54:59,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,633 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:54:59,633 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:59,633 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:54:59,633 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:54:59,634 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,634 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,635 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:54:59,635 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:54:59,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:54:59,636 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:54:59,636 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:54:59,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,637 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:54:59,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:54:59,637 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:54:59,637 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:54:59,637 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:54:59,637 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:54:59,637 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:54:59,637 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:54:59,637 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:54:59,637 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:54:59,654 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a391137acf0048bca4cccde2ba7086f4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205a391137acf0048bca4cccde2ba7086f4 2023-11-27 14:55:00,060 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/2641841008a0406cab64c1a4afe81370 2023-11-27 14:55:00,065 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/2641841008a0406cab64c1a4afe81370 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2641841008a0406cab64c1a4afe81370 2023-11-27 14:55:00,070 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2641841008a0406cab64c1a4afe81370, entries=1, sequenceid=17, filesize=4.9 K 2023-11-27 14:55:00,071 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 434ms, sequenceid=17, compaction requested=true 2023-11-27 14:55:00,071 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:00,071 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:00,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:00,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:00,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:55:00,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:55:00,074 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,074 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:00,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:00,075 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,075 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:00,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:00,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:00,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:00,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:55:00,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fc8d446[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:00,076 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:55:00,076 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,078 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:55:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:55:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:00,078 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,078 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:00,080 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:55:00,080 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,080 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:00,080 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:55:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:00,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:00,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:00,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:55:00,082 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:55:00,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:00,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:00,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,082 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:00,084 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,084 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:00,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,084 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:00,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:00,085 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,086 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:00,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:00,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:00,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:00,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:00,093 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:00,093 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:00,193 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:00,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:00,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:00,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:00,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,201 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:00,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:00,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:55:00,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:55:00,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,203 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:00,203 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:00,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:00,204 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:00,204 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:00,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:00,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:55:00,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:00,205 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:55:00,205 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:00,205 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:55:00,205 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:00,205 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:00,205 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:00,205 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:55:00,205 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:00,619 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512284dcc66d39cb14a298ce35ebd0d0479d9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512284dcc66d39cb14a298ce35ebd0d0479d9 2023-11-27 14:55:01,026 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/a30482bff124481182e010a412bc3e64 2023-11-27 14:55:01,031 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/a30482bff124481182e010a412bc3e64 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/a30482bff124481182e010a412bc3e64 2023-11-27 14:55:01,035 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/a30482bff124481182e010a412bc3e64, entries=1, sequenceid=21, filesize=4.9 K 2023-11-27 14:55:01,036 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 831ms, sequenceid=21, compaction requested=true 2023-11-27 14:55:01,036 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:01,036 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:01,036 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:01,036 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:01,036 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,036 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:55:01,036 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:55:01,038 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,038 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:01,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:01,038 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,038 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:01,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:01,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:01,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:01,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:55:01,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@194c366[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:01,040 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:55:01,040 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,041 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,042 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:55:01,042 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,042 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:01,042 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:55:01,042 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:01,044 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,044 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:55:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:01,044 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:01,044 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:55:01,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:01,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:01,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:01,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:01,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:55:01,047 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:55:01,047 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:01,047 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:01,047 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,047 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:01,051 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,051 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,051 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:01,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:01,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:01,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:01,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:01,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:01,053 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:01,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,054 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:01,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,056 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,056 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,056 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,056 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,057 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,057 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:01,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:55:01,057 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:01,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:01,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:01,057 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,057 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:01,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:01,057 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:01,057 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:01,057 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:01,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:01,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:01,157 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:01,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:01,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:01,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:01,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,165 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:01,165 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:01,165 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:55:01,165 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:55:01,166 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,166 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,168 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,168 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:01,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:01,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:01,169 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:01,169 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:01,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,170 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:55:01,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:01,170 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:55:01,170 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:01,170 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:55:01,170 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:01,170 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:01,170 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:01,170 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:55:01,171 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:01,585 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512318b3fa15b205d4ebcbe7ff77ae20f1db0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512318b3fa15b205d4ebcbe7ff77ae20f1db0 2023-11-27 14:55:01,991 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/c62dc2198a104bb5a65471d3536f4488 2023-11-27 14:55:01,995 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/c62dc2198a104bb5a65471d3536f4488 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c62dc2198a104bb5a65471d3536f4488 2023-11-27 14:55:01,999 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c62dc2198a104bb5a65471d3536f4488, entries=1, sequenceid=25, filesize=4.9 K 2023-11-27 14:55:02,000 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 829ms, sequenceid=25, compaction requested=true 2023-11-27 14:55:02,000 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:02,000 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:02,001 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:02,001 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:02,001 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,001 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:55:02,001 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:55:02,002 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,002 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:02,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:02,003 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,003 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:02,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:02,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:02,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:02,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:55:02,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29afd85e[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:02,004 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:55:02,004 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,005 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,005 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:55:02,006 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:55:02,006 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:02,006 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,006 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:02,008 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,008 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:55:02,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:02,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:02,008 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:02,008 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:55:02,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:02,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:02,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:02,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:02,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:55:02,010 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:55:02,010 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:02,010 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:02,010 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,010 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:02,013 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,013 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:02,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:02,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,013 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:02,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:02,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:02,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:02,014 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:02,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,015 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:02,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,017 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,017 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:02,017 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:02,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:02,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:02,017 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:02,017 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,017 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:02,018 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:02,018 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,018 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:02,018 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:02,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:02,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:02,018 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,018 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,118 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:02,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:02,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:02,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:02,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,125 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:02,125 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:02,125 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:55:02,125 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:55:02,126 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,126 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,127 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:02,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:02,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:02,128 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,128 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:02,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:02,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,129 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,129 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:55:02,129 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,129 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:55:02,129 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:02,129 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:55:02,129 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:02,129 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:02,129 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:02,129 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:55:02,129 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:02,542 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101868542e2ad77422284127e05133c1cae to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101868542e2ad77422284127e05133c1cae 2023-11-27 14:55:02,948 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/123fc7e8cf714f28ab5b3b96af4b3177 2023-11-27 14:55:02,952 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/123fc7e8cf714f28ab5b3b96af4b3177 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/123fc7e8cf714f28ab5b3b96af4b3177 2023-11-27 14:55:02,957 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/123fc7e8cf714f28ab5b3b96af4b3177, entries=1, sequenceid=29, filesize=4.9 K 2023-11-27 14:55:02,958 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 829ms, sequenceid=29, compaction requested=true 2023-11-27 14:55:02,958 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:02,958 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:02,958 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:02,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:02,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:55:02,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:55:02,960 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,960 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:02,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:02,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:02,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:02,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:02,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:55:02,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@128a26f[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:02,962 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:55:02,962 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,965 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,965 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:55:02,965 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:55:02,965 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,965 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:02,965 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:02,967 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:55:02,967 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,967 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:02,967 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:55:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:02,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:02,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:55:02,970 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:55:02,970 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:02,970 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:02,970 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,970 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:02,972 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,972 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:02,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:02,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,972 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:02,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:02,972 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:02,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:02,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:02,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,974 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:02,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:02,976 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:02,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-27 14:55:02,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:02,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:02,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:02,976 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:02,977 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:02,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:02,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:03,077 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:03,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:03,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:03,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:03,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,083 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:03,083 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:03,083 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:55:03,083 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:55:03,084 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,084 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,087 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:03,087 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:03,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:03,087 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,087 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:03,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:03,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,088 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,088 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:55:03,088 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,088 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:55:03,088 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:03,089 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:55:03,089 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:03,089 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:03,089 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:03,089 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:55:03,089 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:03,502 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f7186f4ef1154383ad526afdfaa254c4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103f7186f4ef1154383ad526afdfaa254c4 2023-11-27 14:55:03,907 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/2a0bc747d1554c1580cb7644108af14e 2023-11-27 14:55:03,912 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/2a0bc747d1554c1580cb7644108af14e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2a0bc747d1554c1580cb7644108af14e 2023-11-27 14:55:03,916 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2a0bc747d1554c1580cb7644108af14e, entries=1, sequenceid=33, filesize=4.9 K 2023-11-27 14:55:03,917 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 828ms, sequenceid=33, compaction requested=true 2023-11-27 14:55:03,917 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:03,917 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:03,917 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:03,917 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:03,917 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,917 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:55:03,917 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:55:03,919 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,919 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:03,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:03,920 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:03,920 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:03,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:03,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:03,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:55:03,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@589b3ce8[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:03,921 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:55:03,921 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,925 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,925 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:55:03,925 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:55:03,925 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,925 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:03,925 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:03,929 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,929 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:55:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:03,929 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:03,929 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:55:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:03,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:03,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:03,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:55:03,932 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:55:03,932 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:03,932 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:03,932 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,932 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:03,933 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:03,933 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,934 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:03,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:03,934 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,935 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:03,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,938 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:03,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:03,938 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:03,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:04,039 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:04,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:04,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:04,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:04,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,045 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:04,045 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:04,046 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:55:04,046 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:55:04,046 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,046 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,047 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:04,047 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:04,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:04,048 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,048 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:04,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,049 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:55:04,049 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,049 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:55:04,049 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:04,049 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:55:04,049 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:04,049 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:04,049 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:04,049 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:55:04,049 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:04,462 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090703ec4ef55d604dbf842257b3503b92a1 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090703ec4ef55d604dbf842257b3503b92a1 2023-11-27 14:55:04,867 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/d28ac13033074fd18d1314c5cf2939a3 2023-11-27 14:55:04,873 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/d28ac13033074fd18d1314c5cf2939a3 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/d28ac13033074fd18d1314c5cf2939a3 2023-11-27 14:55:04,877 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/d28ac13033074fd18d1314c5cf2939a3, entries=1, sequenceid=37, filesize=4.9 K 2023-11-27 14:55:04,878 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 829ms, sequenceid=37, compaction requested=true 2023-11-27 14:55:04,878 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:04,878 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:04,878 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:04,878 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:04,878 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,878 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:55:04,878 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:55:04,881 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,881 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:04,881 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,881 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:04,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:04,883 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:55:04,883 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7fb34b6e[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:04,883 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:55:04,883 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,884 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,884 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:55:04,884 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:55:04,884 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:04,884 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,884 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:04,886 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,886 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:55:04,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:04,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:04,886 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:04,886 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:55:04,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:04,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:04,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:04,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:04,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:55:04,888 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:55:04,888 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:04,888 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:04,888 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,888 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:04,890 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,890 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:04,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:04,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,890 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:04,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:04,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:04,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:04,890 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:04,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,892 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:04,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:04,895 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:04,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,895 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:04,895 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:04,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:04,895 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:04,896 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:04,996 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:04,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:05,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:05,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:05,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,003 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:05,003 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:05,003 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-27 14:55:05,003 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-27 14:55:05,003 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,003 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,005 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,005 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:05,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:05,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:05,005 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,005 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:05,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:05,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,006 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-27 14:55:05,006 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-27 14:55:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-27 14:55:05,007 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:05,007 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:05,007 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:05,007 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. started... 2023-11-27 14:55:05,007 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8ae027c1211c228b679773750f57d6ba 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:05,420 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120f0fb4713c02c4530bb37bce7b93b724a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120f0fb4713c02c4530bb37bce7b93b724a 2023-11-27 14:55:05,825 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/f034ba12adcd433c85897bf4a31307fa 2023-11-27 14:55:05,830 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/f034ba12adcd433c85897bf4a31307fa as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/f034ba12adcd433c85897bf4a31307fa 2023-11-27 14:55:05,834 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/f034ba12adcd433c85897bf4a31307fa, entries=1, sequenceid=41, filesize=4.9 K 2023-11-27 14:55:05,835 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8ae027c1211c228b679773750f57d6ba in 828ms, sequenceid=41, compaction requested=true 2023-11-27 14:55:05,835 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:05,835 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:05,835 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:05,835 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:05,835 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,835 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-27 14:55:05,835 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-27 14:55:05,837 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,837 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:05,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:05,837 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,837 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:05,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:05,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:05,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:05,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-27 14:55:05,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@280911ee[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:05,839 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-27 14:55:05,839 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,841 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,841 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-27 14:55:05,841 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-27 14:55:05,841 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:05,841 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,841 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:05,843 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-27 14:55:05,843 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,843 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:05,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:05,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:05,843 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-27 14:55:05,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:05,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:05,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:05,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:05,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:55:05,846 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-27 14:55:05,846 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:05,846 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:05,846 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,846 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:05,847 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,847 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:05,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:05,847 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:05,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:05,848 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:05,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:05,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:05,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,849 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:05,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,854 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,854 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:05,854 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,854 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:05,854 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:05,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:05,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-27 14:55:05,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:05,854 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:05,854 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:05,855 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,855 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,855 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,855 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:05,855 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:05,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:05,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:05,955 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:05,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:05,957 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-11-27 14:55:05,959 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:55:05,960 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:55:05,961 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:05,961 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097505961 2023-11-27 14:55:05,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:55:05,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-11-27 14:55:05,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:55:05,966 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x11129896 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:55:05,971 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@98ca83e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:55:05,971 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:06,818 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036a278ac9f47a4721aac08d2be518d28d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601036a278ac9f47a4721aac08d2be518d28d 2023-11-27 14:55:06,819 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:55:06,836 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:55:06,838 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56728, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:55:06,844 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:55:06,845 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58606, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:55:06,855 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/242d957e897d44eaa470dcaeb60cb3d3 first=Optional[r3] last=Optional[r6] 2023-11-27 14:55:06,856 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/242d957e897d44eaa470dcaeb60cb3d3}] 2023-11-27 14:55:06,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/242d957e897d44eaa470dcaeb60cb3d3 for inclusion in 8ae027c1211c228b679773750f57d6ba/family1 2023-11-27 14:55:06,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-27 14:55:06,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:55:06,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:06,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/242d957e897d44eaa470dcaeb60cb3d3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__3m2m9nsse0a9j8jr0bb35b910i4rgijkchedljvoh7bv59mbtvkkhj4limq3atgc/family1/242d957e897d44eaa470dcaeb60cb3d3 2023-11-27 14:55:06,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__3m2m9nsse0a9j8jr0bb35b910i4rgijkchedljvoh7bv59mbtvkkhj4limq3atgc/family1/242d957e897d44eaa470dcaeb60cb3d3 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7589f152145e41a681f8342271ee1417_SeqId_44_ 2023-11-27 14:55:06,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__3m2m9nsse0a9j8jr0bb35b910i4rgijkchedljvoh7bv59mbtvkkhj4limq3atgc/family1/242d957e897d44eaa470dcaeb60cb3d3 into 8ae027c1211c228b679773750f57d6ba/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7589f152145e41a681f8342271ee1417_SeqId_44_ - updating store file list. 2023-11-27 14:55:06,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7589f152145e41a681f8342271ee1417_SeqId_44_ into 8ae027c1211c228b679773750f57d6ba/family1 2023-11-27 14:55:06,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__3m2m9nsse0a9j8jr0bb35b910i4rgijkchedljvoh7bv59mbtvkkhj4limq3atgc/family1/242d957e897d44eaa470dcaeb60cb3d3 into 8ae027c1211c228b679773750f57d6ba/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7589f152145e41a681f8342271ee1417_SeqId_44_) 2023-11-27 14:55:06,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__3m2m9nsse0a9j8jr0bb35b910i4rgijkchedljvoh7bv59mbtvkkhj4limq3atgc/family1/242d957e897d44eaa470dcaeb60cb3d3 2023-11-27 14:55:06,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-27 14:55:06,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-27 14:55:06,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): 8ae027c1211c228b679773750f57d6ba/family1 is initiating minor compaction 2023-11-27 14:55:06,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2cf37072; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:06,874 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8ae027c1211c228b679773750f57d6ba/family1 in testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:55:06,874 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7fd24fbf1fe24de1ace1788e470fc3fa, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/ab323b443aa24eabb8b2406b93474d07, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2641841008a0406cab64c1a4afe81370, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/a30482bff124481182e010a412bc3e64, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c62dc2198a104bb5a65471d3536f4488, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/123fc7e8cf714f28ab5b3b96af4b3177, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2a0bc747d1554c1580cb7644108af14e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/d28ac13033074fd18d1314c5cf2939a3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/f034ba12adcd433c85897bf4a31307fa, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7589f152145e41a681f8342271ee1417_SeqId_44_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp, totalSize=50.0 K 2023-11-27 14:55:06,874 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 7fd24fbf1fe24de1ace1788e470fc3fa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-27 14:55:06,874 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting ab323b443aa24eabb8b2406b93474d07, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-27 14:55:06,874 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 2641841008a0406cab64c1a4afe81370, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-27 14:55:06,874 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting a30482bff124481182e010a412bc3e64, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-27 14:55:06,874 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting c62dc2198a104bb5a65471d3536f4488, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-27 14:55:06,874 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 123fc7e8cf714f28ab5b3b96af4b3177, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-27 14:55:06,875 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 2a0bc747d1554c1580cb7644108af14e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-27 14:55:06,875 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting d28ac13033074fd18d1314c5cf2939a3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-27 14:55:06,875 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting f034ba12adcd433c85897bf4a31307fa, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-27 14:55:06,875 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 7589f152145e41a681f8342271ee1417_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-27 14:55:06,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:55:06,878 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:06,879 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512284dcc66d39cb14a298ce35ebd0d0479d9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512284dcc66d39cb14a298ce35ebd0d0479d9 2023-11-27 14:55:06,881 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512318b3fa15b205d4ebcbe7ff77ae20f1db0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512318b3fa15b205d4ebcbe7ff77ae20f1db0 2023-11-27 14:55:06,883 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101868542e2ad77422284127e05133c1cae to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101868542e2ad77422284127e05133c1cae 2023-11-27 14:55:06,884 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103f7186f4ef1154383ad526afdfaa254c4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103f7186f4ef1154383ad526afdfaa254c4 2023-11-27 14:55:06,884 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-11-27 14:55:06,884 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-27 14:55:06,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:06,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:06,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:06,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:06,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:06,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:06,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:06,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:06,919 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8ae027c1211c228b679773750f57d6ba#family1#compaction#48 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:55:06,953 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120510b10c3ad9b74450ae4dcb50531d84db to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120510b10c3ad9b74450ae4dcb50531d84db 2023-11-27 14:55:06,954 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:55:06,978 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5e0e529de2e94ebab1fa0e4ce276445b first=Optional[r00] last=Optional[r2] 2023-11-27 14:55:06,979 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5e0e529de2e94ebab1fa0e4ce276445b}] 2023-11-27 14:55:06,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5e0e529de2e94ebab1fa0e4ce276445b for inclusion in 8ae027c1211c228b679773750f57d6ba/family1 2023-11-27 14:55:06,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-27 14:55:06,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:55:06,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:06,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/5e0e529de2e94ebab1fa0e4ce276445b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1dfes3keau3nnr66ognoisnimg2mmkfmt6b0ru1hgjtn5nda35nq89gspjgq65pk/family1/5e0e529de2e94ebab1fa0e4ce276445b 2023-11-27 14:55:06,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1dfes3keau3nnr66ognoisnimg2mmkfmt6b0ru1hgjtn5nda35nq89gspjgq65pk/family1/5e0e529de2e94ebab1fa0e4ce276445b as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c0e93909f98848c899c710da74480a5a_SeqId_46_ 2023-11-27 14:55:06,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1dfes3keau3nnr66ognoisnimg2mmkfmt6b0ru1hgjtn5nda35nq89gspjgq65pk/family1/5e0e529de2e94ebab1fa0e4ce276445b into 8ae027c1211c228b679773750f57d6ba/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c0e93909f98848c899c710da74480a5a_SeqId_46_ - updating store file list. 2023-11-27 14:55:06,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c0e93909f98848c899c710da74480a5a_SeqId_46_ into 8ae027c1211c228b679773750f57d6ba/family1 2023-11-27 14:55:06,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1dfes3keau3nnr66ognoisnimg2mmkfmt6b0ru1hgjtn5nda35nq89gspjgq65pk/family1/5e0e529de2e94ebab1fa0e4ce276445b into 8ae027c1211c228b679773750f57d6ba/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c0e93909f98848c899c710da74480a5a_SeqId_46_) 2023-11-27 14:55:06,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__1dfes3keau3nnr66ognoisnimg2mmkfmt6b0ru1hgjtn5nda35nq89gspjgq65pk/family1/5e0e529de2e94ebab1fa0e4ce276445b 2023-11-27 14:55:06,995 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:55:06,995 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:06,997 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511301358e2c3e19f4144af39b4b3c2d2c47b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511301358e2c3e19f4144af39b4b3c2d2c47b 2023-11-27 14:55:06,998 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120103313e14a6664c38816cd5dffe7940f8 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120103313e14a6664c38816cd5dffe7940f8 2023-11-27 14:55:06,999 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205a391137acf0048bca4cccde2ba7086f4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205a391137acf0048bca4cccde2ba7086f4 2023-11-27 14:55:06,999 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-11-27 14:55:06,999 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-27 14:55:07,000 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-27 14:55:07,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-27 14:55:07,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:07,001 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:55:07,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0430 sec 2023-11-27 14:55:07,112 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=263 (was 257) - Thread LEAK? -, OpenFileDescriptor=658 (was 622) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=286 (was 310), ProcessCount=168 (was 168), AvailableMemoryMB=8243 (was 8250) 2023-11-27 14:55:07,127 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=263, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=168, AvailableMemoryMB=8243 2023-11-27 14:55:07,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-27 14:55:07,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:07,132 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:55:07,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-11-27 14:55:07,133 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:55:07,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-27 14:55:07,135 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:07,135 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8 empty. 2023-11-27 14:55:07,136 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:07,136 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-11-27 14:55:07,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-27 14:55:07,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-27 14:55:07,544 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-11-27 14:55:07,545 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 39f113e3442d78447a6f24b4b73006c8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:55:07,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-27 14:55:07,738 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/.tmp/family1/5951018f55ae455aa3e1f27c2bf910c7 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/5951018f55ae455aa3e1f27c2bf910c7 2023-11-27 14:55:07,744 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 8ae027c1211c228b679773750f57d6ba/family1 of 8ae027c1211c228b679773750f57d6ba into 5951018f55ae455aa3e1f27c2bf910c7(size=6.6 K), total size for store is 337.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:55:07,744 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:55:07,744 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1701096906873; duration=0sec 2023-11-27 14:55:07,744 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:07,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7fd24fbf1fe24de1ace1788e470fc3fa, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/ab323b443aa24eabb8b2406b93474d07, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2641841008a0406cab64c1a4afe81370, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/a30482bff124481182e010a412bc3e64, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c62dc2198a104bb5a65471d3536f4488, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/123fc7e8cf714f28ab5b3b96af4b3177, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2a0bc747d1554c1580cb7644108af14e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/d28ac13033074fd18d1314c5cf2939a3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/f034ba12adcd433c85897bf4a31307fa, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7589f152145e41a681f8342271ee1417_SeqId_44_] to archive 2023-11-27 14:55:07,763 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:07,764 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7fd24fbf1fe24de1ace1788e470fc3fa to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7fd24fbf1fe24de1ace1788e470fc3fa 2023-11-27 14:55:07,765 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/ab323b443aa24eabb8b2406b93474d07 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/ab323b443aa24eabb8b2406b93474d07 2023-11-27 14:55:07,766 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2641841008a0406cab64c1a4afe81370 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2641841008a0406cab64c1a4afe81370 2023-11-27 14:55:07,767 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/a30482bff124481182e010a412bc3e64 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/a30482bff124481182e010a412bc3e64 2023-11-27 14:55:07,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c62dc2198a104bb5a65471d3536f4488 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/c62dc2198a104bb5a65471d3536f4488 2023-11-27 14:55:07,769 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/123fc7e8cf714f28ab5b3b96af4b3177 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/123fc7e8cf714f28ab5b3b96af4b3177 2023-11-27 14:55:07,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2a0bc747d1554c1580cb7644108af14e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/2a0bc747d1554c1580cb7644108af14e 2023-11-27 14:55:07,772 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/d28ac13033074fd18d1314c5cf2939a3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/d28ac13033074fd18d1314c5cf2939a3 2023-11-27 14:55:07,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/f034ba12adcd433c85897bf4a31307fa to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/f034ba12adcd433c85897bf4a31307fa 2023-11-27 14:55:07,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7589f152145e41a681f8342271ee1417_SeqId_44_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/family1/7589f152145e41a681f8342271ee1417_SeqId_44_ 2023-11-27 14:55:07,951 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:07,951 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 39f113e3442d78447a6f24b4b73006c8, disabling compactions & flushes 2023-11-27 14:55:07,951 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:07,951 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:07,951 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. after waiting 0 ms 2023-11-27 14:55:07,951 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:07,951 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:07,951 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:07,953 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:55:07,954 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096907954"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096907954"}]},"ts":"1701096907954"} 2023-11-27 14:55:07,956 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-27 14:55:07,957 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:55:07,957 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096907957"}]},"ts":"1701096907957"} 2023-11-27 14:55:07,958 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-11-27 14:55:07,961 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, ASSIGN}] 2023-11-27 14:55:07,962 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, ASSIGN 2023-11-27 14:55:07,963 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:55:08,114 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:08,114 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096908114"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096908114"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096908114"}]},"ts":"1701096908114"} 2023-11-27 14:55:08,116 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:08,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-27 14:55:08,268 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:08,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:08,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 39f113e3442d78447a6f24b4b73006c8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:55:08,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:08,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:08,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:08,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:08,272 INFO [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:08,274 DEBUG [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:08,274 DEBUG [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:08,274 INFO [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39f113e3442d78447a6f24b4b73006c8 columnFamilyName family1 2023-11-27 14:55:08,275 INFO [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] regionserver.HStore(310): Store=39f113e3442d78447a6f24b4b73006c8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:08,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:08,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:08,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:08,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:55:08,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 39f113e3442d78447a6f24b4b73006c8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64615732, jitterRate=-0.037150561809539795}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:55:08,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:08,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8., pid=41, masterSystemTime=1701096908268 2023-11-27 14:55:08,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:08,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:08,283 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:08,284 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096908283"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096908283"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096908283"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096908283"}]},"ts":"1701096908283"} 2023-11-27 14:55:08,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=41, resume processing ppid=40 2023-11-27 14:55:08,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360 in 169 msec 2023-11-27 14:55:08,288 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=40, resume processing ppid=39 2023-11-27 14:55:08,288 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, ASSIGN in 326 msec 2023-11-27 14:55:08,288 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:55:08,289 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096908289"}]},"ts":"1701096908289"} 2023-11-27 14:55:08,290 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-11-27 14:55:08,292 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:55:08,293 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1630 sec 2023-11-27 14:55:09,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-27 14:55:09,239 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-11-27 14:55:09,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:09,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:09,271 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:09,271 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:09,271 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:09,271 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:09,271 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:09,271 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:09,273 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:09,273 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:09,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:09,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:09,273 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:09,273 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:09,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:09,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:09,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:09,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:09,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:09,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:09,274 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:09,274 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:09,274 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:09,275 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:09,275 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:09,275 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:09,275 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:09,275 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-27 14:55:09,695 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b987317b621446029a3a8ec19c17cdce to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130b987317b621446029a3a8ec19c17cdce 2023-11-27 14:55:10,099 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/c409682ea72644ea9992887fc4f3cb4e 2023-11-27 14:55:10,104 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/c409682ea72644ea9992887fc4f3cb4e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/c409682ea72644ea9992887fc4f3cb4e 2023-11-27 14:55:10,110 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/c409682ea72644ea9992887fc4f3cb4e, entries=1000, sequenceid=5, filesize=191.4 K 2023-11-27 14:55:10,111 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 836ms, sequenceid=5, compaction requested=false 2023-11-27 14:55:10,111 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:10,111 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:10,111 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:10,111 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:10,111 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:10,111 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,111 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:10,111 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:10,113 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,113 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:10,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:10,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:10,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:10,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:10,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:10,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:10,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cf6f55a[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:10,115 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:10,115 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,116 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,116 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:10,116 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:10,116 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:10,116 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,117 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:10,118 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:10,118 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:10,118 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,118 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:10,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:10,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:10,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:10,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:10,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:10,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:10,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:10,121 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:10,121 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:10,121 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:10,121 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,121 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:10,122 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,122 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:10,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,123 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:10,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:10,123 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:10,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,124 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:10,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,127 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:10,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:10,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:10,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:10,127 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:10,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:10,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:10,227 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:10,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:10,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:10,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:10,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,235 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:10,235 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:10,235 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:10,235 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:10,235 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,236 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,237 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,237 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:10,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:10,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:10,237 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:10,238 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:10,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:10,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-27 14:55:10,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:10,239 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:10,239 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:10,239 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:10,239 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:10,239 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:10,239 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:10,239 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:10,239 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-27 14:55:10,654 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c0ce32df9b5c4e939d4b23e8a9f0b4b2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128c0ce32df9b5c4e939d4b23e8a9f0b4b2 2023-11-27 14:55:11,060 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/33f095d249844113a65df109e3f3e0c5 2023-11-27 14:55:11,065 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/33f095d249844113a65df109e3f3e0c5 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/33f095d249844113a65df109e3f3e0c5 2023-11-27 14:55:11,070 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/33f095d249844113a65df109e3f3e0c5, entries=1, sequenceid=9, filesize=5.0 K 2023-11-27 14:55:11,071 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 832ms, sequenceid=9, compaction requested=false 2023-11-27 14:55:11,071 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:11,071 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:11,074 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,074 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:11,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:11,074 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,075 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:11,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:11,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:11,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:11,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:11,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d0be850[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:11,076 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:11,076 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,077 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,077 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:11,077 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:11,077 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:11,077 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,078 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:11,079 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:11,079 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,079 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:11,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:11,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:11,079 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:11,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:11,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:11,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:11,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:11,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:11,082 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:11,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:11,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:11,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,082 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:11,083 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,083 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:11,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:11,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,083 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:11,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:11,084 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:11,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:11,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:11,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,085 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:11,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,092 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:11,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:11,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:11,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:11,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:11,092 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:11,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:11,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:11,192 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:11,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:11,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:11,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:11,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,199 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:11,199 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:11,199 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:11,199 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:11,200 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,200 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,201 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:11,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:11,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:11,202 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:11,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,202 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:11,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:11,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-27 14:55:11,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:11,203 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:11,203 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:11,203 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:11,203 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:11,203 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:11,203 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:11,203 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:11,203 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:11,617 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201a1f46f51b3164571b363bd71be4acfd6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201a1f46f51b3164571b363bd71be4acfd6 2023-11-27 14:55:11,645 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.07692307692307693, tune throughput to 53.85 MB/second 2023-11-27 14:55:12,023 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/86f12ee684bd4d58997d5b2fd01fe858 2023-11-27 14:55:12,029 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/86f12ee684bd4d58997d5b2fd01fe858 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/86f12ee684bd4d58997d5b2fd01fe858 2023-11-27 14:55:12,033 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/86f12ee684bd4d58997d5b2fd01fe858, entries=1, sequenceid=13, filesize=5.0 K 2023-11-27 14:55:12,033 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 830ms, sequenceid=13, compaction requested=true 2023-11-27 14:55:12,034 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:12,034 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:12,034 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:12,034 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:12,034 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,034 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:12,034 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:12,035 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,035 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:12,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:12,036 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,036 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:12,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:12,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:12,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:12,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@88636a4[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:12,038 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:12,038 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,040 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:12,040 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,040 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:12,041 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:12,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:12,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:12,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:12,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:12,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:12,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:12,044 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:12,044 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:12,044 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:12,044 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:12,044 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,044 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:12,046 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,046 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:12,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:12,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,046 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:12,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:12,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:12,046 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:12,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:12,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,048 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:12,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,050 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,050 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,050 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,050 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,050 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,050 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:12,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-27 14:55:12,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:12,050 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:12,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:12,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:12,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:12,050 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:12,050 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,050 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:12,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:12,051 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:12,051 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:12,151 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:12,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:12,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:12,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:12,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,158 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:12,158 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:12,158 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:12,158 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:12,158 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,158 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,160 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:12,160 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:12,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:12,160 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,160 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:12,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:12,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,161 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,161 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-27 14:55:12,161 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,161 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:12,161 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:12,161 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:12,161 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:12,161 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:12,161 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:12,162 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:12,162 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:12,575 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e66cc73a18504c0e92976acf55976a99 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205e66cc73a18504c0e92976acf55976a99 2023-11-27 14:55:12,981 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/43084449cf714498b35086a0f97dc144 2023-11-27 14:55:12,987 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/43084449cf714498b35086a0f97dc144 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/43084449cf714498b35086a0f97dc144 2023-11-27 14:55:12,992 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/43084449cf714498b35086a0f97dc144, entries=1, sequenceid=17, filesize=5.0 K 2023-11-27 14:55:12,993 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 831ms, sequenceid=17, compaction requested=true 2023-11-27 14:55:12,993 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:12,993 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:12,993 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:12,993 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:12,993 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,993 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:12,993 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:12,995 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:12,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:12,995 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,995 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:12,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:12,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6860d03b[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:12,997 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:12,997 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,998 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:12,998 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:12,998 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:12,998 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:12,999 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:12,999 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:13,001 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:13,001 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:13,001 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,001 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:13,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:13,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:13,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:13,003 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:13,004 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:13,004 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:13,004 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,004 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:13,005 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,005 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:13,005 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:13,006 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,007 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:13,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:13,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:13,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:13,010 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:13,010 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:13,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:13,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:13,110 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:13,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:13,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:13,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:13,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,118 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:13,118 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:13,118 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:13,118 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:13,118 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,118 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,120 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,120 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:13,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:13,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:13,120 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,120 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:13,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,121 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,121 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-27 14:55:13,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,121 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:13,121 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:13,121 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:13,121 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:13,121 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:13,121 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:13,122 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:13,122 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:13,534 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122864672a9e748b4328b40f309c5f7fbc25 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122864672a9e748b4328b40f309c5f7fbc25 2023-11-27 14:55:13,818 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:55:13,939 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/84c83a0100c7450d8f3dded6c3947f1b 2023-11-27 14:55:13,944 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/84c83a0100c7450d8f3dded6c3947f1b as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/84c83a0100c7450d8f3dded6c3947f1b 2023-11-27 14:55:13,948 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/84c83a0100c7450d8f3dded6c3947f1b, entries=1, sequenceid=21, filesize=5.0 K 2023-11-27 14:55:13,949 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 827ms, sequenceid=21, compaction requested=true 2023-11-27 14:55:13,949 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:13,949 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:13,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:13,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:13,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:13,949 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:13,954 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:13,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:13,955 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,955 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:13,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:13,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:13,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:13,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:13,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3367a448[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:13,956 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:13,956 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,957 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,958 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:13,958 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:13,958 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:13,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:13,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:13,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:13,959 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:13,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:13,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:13,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:13,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:13,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:13,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:13,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:13,962 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:13,962 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:13,962 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:13,962 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,962 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:13,963 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,963 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:13,963 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:13,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:13,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:13,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:13,964 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:13,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:13,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,965 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:13,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:13,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:13,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:13,969 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:13,969 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:13,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-27 14:55:13,969 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:13,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:13,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:13,969 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,969 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:13,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:13,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:13,969 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:13,969 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:14,069 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:14,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:14,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:14,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:14,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,076 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:14,076 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:14,076 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:14,076 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:14,077 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,077 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,078 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:14,078 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:14,079 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,079 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,079 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,079 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,080 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-27 14:55:14,080 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,080 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:14,080 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:14,080 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:14,080 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:14,080 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:14,080 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:14,080 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:14,080 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:14,493 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123116878fd30cde47a5b27c0312a368ac98 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123116878fd30cde47a5b27c0312a368ac98 2023-11-27 14:55:14,898 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/aae4257b48b44cf1a3276187c5b8ce01 2023-11-27 14:55:14,903 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/aae4257b48b44cf1a3276187c5b8ce01 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/aae4257b48b44cf1a3276187c5b8ce01 2023-11-27 14:55:14,907 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/aae4257b48b44cf1a3276187c5b8ce01, entries=1, sequenceid=25, filesize=5.0 K 2023-11-27 14:55:14,908 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 828ms, sequenceid=25, compaction requested=true 2023-11-27 14:55:14,908 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:14,908 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:14,910 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,911 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:14,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:14,911 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,911 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:14,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:14,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:14,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:14,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:14,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11c831c[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:14,913 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:14,913 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,914 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,914 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:14,914 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:14,914 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:14,914 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,914 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:14,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,921 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:14,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:14,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:14,921 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:14,921 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:14,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:14,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:14,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:14,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:14,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:14,923 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:14,923 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:14,923 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:14,923 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,923 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:14,925 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:14,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:14,925 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,925 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:14,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:14,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:14,927 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:14,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,931 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:14,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:14,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:14,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:14,931 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:14,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:14,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:15,031 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:15,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:15,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:15,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:15,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,038 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:15,038 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:15,038 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:15,038 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:15,038 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,038 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,040 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:15,040 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:15,040 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,040 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:15,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,041 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,041 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-27 14:55:15,041 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,041 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:15,041 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:15,041 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:15,041 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:15,041 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:15,041 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:15,041 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:15,041 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:15,454 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601015abae85cb34b498a9e5442c729a5bdce to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601015abae85cb34b498a9e5442c729a5bdce 2023-11-27 14:55:15,859 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/b2a10eab810444cb9a87627a6e05e1c9 2023-11-27 14:55:15,864 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/b2a10eab810444cb9a87627a6e05e1c9 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/b2a10eab810444cb9a87627a6e05e1c9 2023-11-27 14:55:15,867 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/b2a10eab810444cb9a87627a6e05e1c9, entries=1, sequenceid=29, filesize=5.0 K 2023-11-27 14:55:15,868 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 827ms, sequenceid=29, compaction requested=true 2023-11-27 14:55:15,868 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:15,868 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:15,868 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:15,868 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:15,869 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,869 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:15,869 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:15,870 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,870 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:15,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:15,870 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,870 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:15,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:15,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:15,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:15,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:15,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3816a68d[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:15,872 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:15,872 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,873 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,873 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:15,873 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:15,873 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:15,874 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,874 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:15,880 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,880 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,880 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:15,880 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:15,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:15,882 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:15,882 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:15,882 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:15,882 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,882 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:15,884 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,884 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:15,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,884 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:15,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:15,884 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:15,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,886 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:15,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:15,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:15,888 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:15,888 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:15,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:15,989 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:15,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:15,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:15,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:15,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,995 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:15,995 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:15,995 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:15,995 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:15,996 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,996 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,997 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:15,997 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:15,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:15,997 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:15,998 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:15,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,998 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:15,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,998 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,998 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-27 14:55:15,999 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:15,999 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:15,999 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:15,999 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:15,999 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:15,999 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:15,999 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:15,999 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:15,999 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:16,412 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038985c8538ec646698080be943bef38b4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601038985c8538ec646698080be943bef38b4 2023-11-27 14:55:16,821 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/ce389eef6da943418c1b2926219ebc13 2023-11-27 14:55:16,826 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/ce389eef6da943418c1b2926219ebc13 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ce389eef6da943418c1b2926219ebc13 2023-11-27 14:55:16,830 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ce389eef6da943418c1b2926219ebc13, entries=1, sequenceid=33, filesize=5.0 K 2023-11-27 14:55:16,830 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 831ms, sequenceid=33, compaction requested=true 2023-11-27 14:55:16,831 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:16,831 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:16,831 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:16,831 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:16,831 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,831 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:16,831 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:16,834 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,834 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:16,834 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,834 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:16,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:16,835 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:16,835 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@569948bc[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:16,835 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:16,836 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,837 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,837 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:16,837 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:16,837 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:16,837 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,837 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:16,838 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,838 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:16,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:16,838 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:16,838 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:16,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:16,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:16,841 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:16,841 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:16,841 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:16,841 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,841 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:16,843 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,843 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:16,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:16,843 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:16,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:16,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:16,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:16,843 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:16,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,845 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:16,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,848 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,848 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,848 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:16,848 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,848 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,848 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:16,849 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:16,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:16,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:16,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-27 14:55:16,848 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:16,849 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:16,849 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,849 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:16,849 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:16,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:16,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:16,949 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:16,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:16,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:16,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:16,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,956 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:16,956 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:16,956 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:16,956 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:16,956 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,956 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,958 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:16,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:16,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:16,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,958 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:16,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,959 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,959 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-27 14:55:16,959 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:16,959 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:16,959 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:16,960 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:17,375 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090729987755510b4c12903bbaf43264fe62 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015090729987755510b4c12903bbaf43264fe62 2023-11-27 14:55:17,780 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/9f46a5f8ed124765851f33c55d6c8ff7 2023-11-27 14:55:17,786 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/9f46a5f8ed124765851f33c55d6c8ff7 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/9f46a5f8ed124765851f33c55d6c8ff7 2023-11-27 14:55:17,790 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/9f46a5f8ed124765851f33c55d6c8ff7, entries=1, sequenceid=37, filesize=5.0 K 2023-11-27 14:55:17,791 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 831ms, sequenceid=37, compaction requested=true 2023-11-27 14:55:17,791 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:17,791 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:17,791 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:17,791 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:17,791 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,791 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:17,791 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:17,793 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,793 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:17,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:17,793 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,793 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:17,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:17,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:17,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:17,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:17,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58d2e3c3[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:17,795 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:17,795 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,796 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,796 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:17,796 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:17,796 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,796 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:17,796 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:17,799 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,799 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:17,799 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:17,799 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:17,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:17,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:17,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:17,801 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:17,801 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:17,801 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:17,801 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,801 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:17,803 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,803 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:17,803 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:17,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:17,803 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,805 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,809 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,809 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,809 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:17,809 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:17,810 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:17,810 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-27 14:55:17,810 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:17,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:17,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:17,810 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,810 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,810 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,810 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,810 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:17,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:17,810 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:17,910 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:17,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:17,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:17,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:17,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,917 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:17,917 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:17,917 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-27 14:55:17,917 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-27 14:55:17,918 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,918 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,920 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:17,920 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:17,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:17,921 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:17,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,921 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:17,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:17,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,922 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-27 14:55:17,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:17,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-27 14:55:17,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:17,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-27 14:55:17,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:17,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:17,922 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:17,922 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. started... 2023-11-27 14:55:17,923 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39f113e3442d78447a6f24b4b73006c8 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:18,338 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e9359edd22fc4f76a3c790a330456fd2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120e9359edd22fc4f76a3c790a330456fd2 2023-11-27 14:55:18,744 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/5508862fa17e4ce7bc4d8798d75a00c9 2023-11-27 14:55:18,749 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/5508862fa17e4ce7bc4d8798d75a00c9 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/5508862fa17e4ce7bc4d8798d75a00c9 2023-11-27 14:55:18,753 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/5508862fa17e4ce7bc4d8798d75a00c9, entries=1, sequenceid=41, filesize=5.0 K 2023-11-27 14:55:18,754 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39f113e3442d78447a6f24b4b73006c8 in 831ms, sequenceid=41, compaction requested=true 2023-11-27 14:55:18,754 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:18,754 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:18,755 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:18,755 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:18,755 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,755 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-27 14:55:18,755 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-27 14:55:18,756 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,756 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:18,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:18,757 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,757 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:18,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:18,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:18,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:18,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-27 14:55:18,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d338fa6[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:18,758 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-27 14:55:18,758 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,759 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,759 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-27 14:55:18,759 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-27 14:55:18,759 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:18,759 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,760 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:18,761 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,761 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-27 14:55:18,761 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:18,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,761 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-27 14:55:18,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:18,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:18,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:18,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:18,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:18,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:18,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-27 14:55:18,763 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-27 14:55:18,763 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:18,763 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:18,763 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,763 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:18,766 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,766 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:18,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:18,766 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:18,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:18,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:18,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:18,767 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:18,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,768 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:18,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,770 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,770 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:18,770 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,770 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:18,770 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:18,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:18,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:18,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:18,770 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:18,770 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:18,771 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:18,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:18,771 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,771 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:18,771 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:18,771 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,871 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:18,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:18,873 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:18,875 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-27 14:55:18,876 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-27 14:55:18,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:18,877 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097518877 2023-11-27 14:55:18,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:55:18,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-27 14:55:18,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:18,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x67971a8b to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:55:18,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@db4f116, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:55:18,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:19,320 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031c6dc664122a4ed889e8af47f82d2f69 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601031c6dc664122a4ed889e8af47f82d2f69 2023-11-27 14:55:19,321 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:19,334 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:55:19,335 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43298, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:55:19,342 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:55:19,344 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35224, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:55:19,356 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/be1f93a560b24ebfa763e4d9bee7e3fa first=Optional[r3] last=Optional[r6] 2023-11-27 14:55:19,356 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/be1f93a560b24ebfa763e4d9bee7e3fa}] 2023-11-27 14:55:19,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/be1f93a560b24ebfa763e4d9bee7e3fa for inclusion in 39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:19,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-27 14:55:19,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:55:19,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:19,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/be1f93a560b24ebfa763e4d9bee7e3fa to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anht3as5otuuimcc0s4mko1s97lvp60l0mcc48af678tnndslqt677dbrvm24asa/family1/be1f93a560b24ebfa763e4d9bee7e3fa 2023-11-27 14:55:19,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anht3as5otuuimcc0s4mko1s97lvp60l0mcc48af678tnndslqt677dbrvm24asa/family1/be1f93a560b24ebfa763e4d9bee7e3fa as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ebbf4c572b5749deb1b0d7252f2c9024_SeqId_44_ 2023-11-27 14:55:19,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anht3as5otuuimcc0s4mko1s97lvp60l0mcc48af678tnndslqt677dbrvm24asa/family1/be1f93a560b24ebfa763e4d9bee7e3fa into 39f113e3442d78447a6f24b4b73006c8/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ebbf4c572b5749deb1b0d7252f2c9024_SeqId_44_ - updating store file list. 2023-11-27 14:55:19,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ebbf4c572b5749deb1b0d7252f2c9024_SeqId_44_ into 39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:19,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anht3as5otuuimcc0s4mko1s97lvp60l0mcc48af678tnndslqt677dbrvm24asa/family1/be1f93a560b24ebfa763e4d9bee7e3fa into 39f113e3442d78447a6f24b4b73006c8/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ebbf4c572b5749deb1b0d7252f2c9024_SeqId_44_) 2023-11-27 14:55:19,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__anht3as5otuuimcc0s4mko1s97lvp60l0mcc48af678tnndslqt677dbrvm24asa/family1/be1f93a560b24ebfa763e4d9bee7e3fa 2023-11-27 14:55:19,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-27 14:55:19,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-27 14:55:19,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): 39f113e3442d78447a6f24b4b73006c8/family1 is initiating minor compaction 2023-11-27 14:55:19,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@434119f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:19,375 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 39f113e3442d78447a6f24b4b73006c8/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:19,375 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/33f095d249844113a65df109e3f3e0c5, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/86f12ee684bd4d58997d5b2fd01fe858, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/43084449cf714498b35086a0f97dc144, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/84c83a0100c7450d8f3dded6c3947f1b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/aae4257b48b44cf1a3276187c5b8ce01, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/b2a10eab810444cb9a87627a6e05e1c9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ce389eef6da943418c1b2926219ebc13, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/9f46a5f8ed124765851f33c55d6c8ff7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/5508862fa17e4ce7bc4d8798d75a00c9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ebbf4c572b5749deb1b0d7252f2c9024_SeqId_44_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp, totalSize=50.4 K 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 33f095d249844113a65df109e3f3e0c5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 86f12ee684bd4d58997d5b2fd01fe858, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 43084449cf714498b35086a0f97dc144, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 84c83a0100c7450d8f3dded6c3947f1b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting aae4257b48b44cf1a3276187c5b8ce01, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting b2a10eab810444cb9a87627a6e05e1c9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting ce389eef6da943418c1b2926219ebc13, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 9f46a5f8ed124765851f33c55d6c8ff7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 5508862fa17e4ce7bc4d8798d75a00c9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-27 14:55:19,375 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting ebbf4c572b5749deb1b0d7252f2c9024_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-27 14:55:19,377 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:19,377 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:19,379 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122864672a9e748b4328b40f309c5f7fbc25 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122864672a9e748b4328b40f309c5f7fbc25 2023-11-27 14:55:19,380 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123116878fd30cde47a5b27c0312a368ac98 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123116878fd30cde47a5b27c0312a368ac98 2023-11-27 14:55:19,381 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601015abae85cb34b498a9e5442c729a5bdce to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601015abae85cb34b498a9e5442c729a5bdce 2023-11-27 14:55:19,382 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601038985c8538ec646698080be943bef38b4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601038985c8538ec646698080be943bef38b4 2023-11-27 14:55:19,382 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-11-27 14:55:19,382 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-27 14:55:19,383 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:19,383 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:19,383 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:19,383 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:19,383 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:19,383 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:19,383 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:19,383 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:19,406 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 39f113e3442d78447a6f24b4b73006c8#family1#compaction#59 average throughput is 1.59 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:55:19,850 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/10e67fee0026442f8024c55b3782bf59 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/10e67fee0026442f8024c55b3782bf59 2023-11-27 14:55:19,855 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 39f113e3442d78447a6f24b4b73006c8/family1 of 39f113e3442d78447a6f24b4b73006c8 into 10e67fee0026442f8024c55b3782bf59(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:55:19,855 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:19,855 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1701096919375; duration=0sec 2023-11-27 14:55:19,855 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:19,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/33f095d249844113a65df109e3f3e0c5, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/86f12ee684bd4d58997d5b2fd01fe858, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/43084449cf714498b35086a0f97dc144, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/84c83a0100c7450d8f3dded6c3947f1b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/aae4257b48b44cf1a3276187c5b8ce01, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/b2a10eab810444cb9a87627a6e05e1c9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ce389eef6da943418c1b2926219ebc13, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/9f46a5f8ed124765851f33c55d6c8ff7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/5508862fa17e4ce7bc4d8798d75a00c9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ebbf4c572b5749deb1b0d7252f2c9024_SeqId_44_] to archive 2023-11-27 14:55:19,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:19,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/33f095d249844113a65df109e3f3e0c5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/33f095d249844113a65df109e3f3e0c5 2023-11-27 14:55:19,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/86f12ee684bd4d58997d5b2fd01fe858 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/86f12ee684bd4d58997d5b2fd01fe858 2023-11-27 14:55:19,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/43084449cf714498b35086a0f97dc144 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/43084449cf714498b35086a0f97dc144 2023-11-27 14:55:19,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/84c83a0100c7450d8f3dded6c3947f1b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/84c83a0100c7450d8f3dded6c3947f1b 2023-11-27 14:55:19,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/aae4257b48b44cf1a3276187c5b8ce01 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/aae4257b48b44cf1a3276187c5b8ce01 2023-11-27 14:55:19,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/b2a10eab810444cb9a87627a6e05e1c9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/b2a10eab810444cb9a87627a6e05e1c9 2023-11-27 14:55:19,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ce389eef6da943418c1b2926219ebc13 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ce389eef6da943418c1b2926219ebc13 2023-11-27 14:55:19,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/9f46a5f8ed124765851f33c55d6c8ff7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/9f46a5f8ed124765851f33c55d6c8ff7 2023-11-27 14:55:19,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/5508862fa17e4ce7bc4d8798d75a00c9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/5508862fa17e4ce7bc4d8798d75a00c9 2023-11-27 14:55:19,886 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ebbf4c572b5749deb1b0d7252f2c9024_SeqId_44_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/ebbf4c572b5749deb1b0d7252f2c9024_SeqId_44_ 2023-11-27 14:55:20,250 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d13a204c738f4a838dcbc463f279b8ec to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d13a204c738f4a838dcbc463f279b8ec 2023-11-27 14:55:20,251 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:20,276 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb5273f3da0d4e5e8413b1057d58b638 first=Optional[r00] last=Optional[r2] 2023-11-27 14:55:20,276 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb5273f3da0d4e5e8413b1057d58b638}] 2023-11-27 14:55:20,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb5273f3da0d4e5e8413b1057d58b638 for inclusion in 39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:20,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-27 14:55:20,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:55:20,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:20,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb5273f3da0d4e5e8413b1057d58b638 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hoqh8mar8chptna12svnaiuoi5l6sget9pq6v2trmohom6ragp1jskde772378i9/family1/bb5273f3da0d4e5e8413b1057d58b638 2023-11-27 14:55:20,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hoqh8mar8chptna12svnaiuoi5l6sget9pq6v2trmohom6ragp1jskde772378i9/family1/bb5273f3da0d4e5e8413b1057d58b638 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/f24fd911669a48c799a734b58344c68d_SeqId_47_ 2023-11-27 14:55:20,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hoqh8mar8chptna12svnaiuoi5l6sget9pq6v2trmohom6ragp1jskde772378i9/family1/bb5273f3da0d4e5e8413b1057d58b638 into 39f113e3442d78447a6f24b4b73006c8/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/f24fd911669a48c799a734b58344c68d_SeqId_47_ - updating store file list. 2023-11-27 14:55:20,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/f24fd911669a48c799a734b58344c68d_SeqId_47_ into 39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:20,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hoqh8mar8chptna12svnaiuoi5l6sget9pq6v2trmohom6ragp1jskde772378i9/family1/bb5273f3da0d4e5e8413b1057d58b638 into 39f113e3442d78447a6f24b4b73006c8/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/f24fd911669a48c799a734b58344c68d_SeqId_47_) 2023-11-27 14:55:20,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hoqh8mar8chptna12svnaiuoi5l6sget9pq6v2trmohom6ragp1jskde772378i9/family1/bb5273f3da0d4e5e8413b1057d58b638 2023-11-27 14:55:20,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-27 14:55:20,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-27 14:55:20,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): 39f113e3442d78447a6f24b4b73006c8/family1 is initiating minor compaction (all files) 2023-11-27 14:55:20,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@487e0464; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:20,293 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 39f113e3442d78447a6f24b4b73006c8/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:20,293 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/c409682ea72644ea9992887fc4f3cb4e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/10e67fee0026442f8024c55b3782bf59, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/f24fd911669a48c799a734b58344c68d_SeqId_47_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp, totalSize=390.1 K 2023-11-27 14:55:20,293 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting c409682ea72644ea9992887fc4f3cb4e, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878813266 2023-11-27 14:55:20,294 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 10e67fee0026442f8024c55b3782bf59, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621213266 2023-11-27 14:55:20,294 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting f24fd911669a48c799a734b58344c68d_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878813266 2023-11-27 14:55:20,295 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:20,295 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:20,297 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130b987317b621446029a3a8ec19c17cdce to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130b987317b621446029a3a8ec19c17cdce 2023-11-27 14:55:20,298 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201a1f46f51b3164571b363bd71be4acfd6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201a1f46f51b3164571b363bd71be4acfd6 2023-11-27 14:55:20,300 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205e66cc73a18504c0e92976acf55976a99 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205e66cc73a18504c0e92976acf55976a99 2023-11-27 14:55:20,300 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-11-27 14:55:20,300 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-27 14:55:20,300 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:20,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:20,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:20,302 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-27 14:55:20,303 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.4280 sec 2023-11-27 14:55:20,316 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 39f113e3442d78447a6f24b4b73006c8#family1#compaction#60 average throughput is 15.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:55:20,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-27 14:55:20,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:20,795 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-27 14:55:20,796 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-11-27 14:55:20,796 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-27 14:55:20,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-11-27 14:55:20,801 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, REOPEN/MOVE}] 2023-11-27 14:55:20,802 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, REOPEN/MOVE 2023-11-27 14:55:20,803 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:20,803 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096920803"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096920803"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096920803"}]},"ts":"1701096920803"} 2023-11-27 14:55:20,804 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:20,956 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:20,956 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:20,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 39f113e3442d78447a6f24b4b73006c8, disabling compactions & flushes 2023-11-27 14:55:20,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:21,536 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/.tmp/family1/5854fac4e1884090a4abcbc7c04d7444 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/5854fac4e1884090a4abcbc7c04d7444 2023-11-27 14:55:21,541 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 39f113e3442d78447a6f24b4b73006c8/family1 of 39f113e3442d78447a6f24b4b73006c8 into 5854fac4e1884090a4abcbc7c04d7444(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:55:21,541 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:21,541 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:21,541 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1701096920292; duration=1sec 2023-11-27 14:55:21,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:21,541 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:21,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. after waiting 0 ms 2023-11-27 14:55:21,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:21,543 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/c409682ea72644ea9992887fc4f3cb4e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/10e67fee0026442f8024c55b3782bf59, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/f24fd911669a48c799a734b58344c68d_SeqId_47_] to archive 2023-11-27 14:55:21,544 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:21,545 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/c409682ea72644ea9992887fc4f3cb4e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/c409682ea72644ea9992887fc4f3cb4e 2023-11-27 14:55:21,546 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/10e67fee0026442f8024c55b3782bf59 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/10e67fee0026442f8024c55b3782bf59 2023-11-27 14:55:21,548 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/f24fd911669a48c799a734b58344c68d_SeqId_47_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/f24fd911669a48c799a734b58344c68d_SeqId_47_ 2023-11-27 14:55:21,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-27 14:55:21,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:21,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:21,553 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: 39f113e3442d78447a6f24b4b73006c8 to self. 2023-11-27 14:55:21,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:21,555 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=CLOSED 2023-11-27 14:55:21,555 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096921555"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096921555"}]},"ts":"1701096921555"} 2023-11-27 14:55:21,558 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=46, resume processing ppid=45 2023-11-27 14:55:21,559 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360 in 753 msec 2023-11-27 14:55:21,559 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=true 2023-11-27 14:55:21,709 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:21,709 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096921709"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096921709"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096921709"}]},"ts":"1701096921709"} 2023-11-27 14:55:21,711 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:21,863 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:21,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:21,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 39f113e3442d78447a6f24b4b73006c8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:55:21,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:21,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:21,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:21,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:21,867 INFO [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:21,868 DEBUG [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:21,868 DEBUG [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:21,869 INFO [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39f113e3442d78447a6f24b4b73006c8 columnFamilyName family1 2023-11-27 14:55:21,874 DEBUG [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/5854fac4e1884090a4abcbc7c04d7444 2023-11-27 14:55:21,874 INFO [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] regionserver.HStore(310): Store=39f113e3442d78447a6f24b4b73006c8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:21,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:21,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:21,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:21,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 39f113e3442d78447a6f24b4b73006c8; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61188887, jitterRate=-0.08821453154087067}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:55:21,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:21,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8., pid=47, masterSystemTime=1701096921862 2023-11-27 14:55:21,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:21,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:21,882 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:21,882 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096921882"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096921882"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096921882"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096921882"}]},"ts":"1701096921882"} 2023-11-27 14:55:21,884 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=47, resume processing ppid=45 2023-11-27 14:55:21,885 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360 in 172 msec 2023-11-27 14:55:21,886 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=45, resume processing ppid=44 2023-11-27 14:55:21,886 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, REOPEN/MOVE in 1.0840 sec 2023-11-27 14:55:21,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=44, resume processing ppid=43 2023-11-27 14:55:21,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0880 sec 2023-11-27 14:55:21,889 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5030 sec 2023-11-27 14:55:21,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=43 2023-11-27 14:55:21,891 INFO [Listener at localhost/37439] compactions.TestMobCompactor(278): alter status finished 2023-11-27 14:55:21,892 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:21,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-27 14:55:21,894 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-27 14:55:21,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:21,894 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097521894 2023-11-27 14:55:21,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:55:21,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-27 14:55:21,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:21,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x489a311b to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:55:21,905 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@620c455f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:55:21,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:21,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:21,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:21,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:21,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:22,735 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289ca867e629384895a677f996ac3eb967 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511289ca867e629384895a677f996ac3eb967 2023-11-27 14:55:22,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:22,749 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:55:22,751 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33368, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:55:22,757 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:55:22,759 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47954, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:55:22,768 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/fae569bc02ae4f27b64b648ffd6d17b3 first=Optional[r05] last=Optional[r8] 2023-11-27 14:55:22,768 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=51 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/fae569bc02ae4f27b64b648ffd6d17b3}] 2023-11-27 14:55:22,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/fae569bc02ae4f27b64b648ffd6d17b3 for inclusion in 39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:22,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-27 14:55:22,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:55:22,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:22,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/fae569bc02ae4f27b64b648ffd6d17b3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p5vu2n4fpl6b83rocnn9f32tsujitalji47hc1mcs7aqs8orsu10rul45r28i0a9/family1/fae569bc02ae4f27b64b648ffd6d17b3 2023-11-27 14:55:22,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p5vu2n4fpl6b83rocnn9f32tsujitalji47hc1mcs7aqs8orsu10rul45r28i0a9/family1/fae569bc02ae4f27b64b648ffd6d17b3 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/94739b77e35440599c0891206dc25c92_SeqId_53_ 2023-11-27 14:55:22,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p5vu2n4fpl6b83rocnn9f32tsujitalji47hc1mcs7aqs8orsu10rul45r28i0a9/family1/fae569bc02ae4f27b64b648ffd6d17b3 into 39f113e3442d78447a6f24b4b73006c8/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/94739b77e35440599c0891206dc25c92_SeqId_53_ - updating store file list. 2023-11-27 14:55:22,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/94739b77e35440599c0891206dc25c92_SeqId_53_ into 39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:22,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p5vu2n4fpl6b83rocnn9f32tsujitalji47hc1mcs7aqs8orsu10rul45r28i0a9/family1/fae569bc02ae4f27b64b648ffd6d17b3 into 39f113e3442d78447a6f24b4b73006c8/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/94739b77e35440599c0891206dc25c92_SeqId_53_) 2023-11-27 14:55:22,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__p5vu2n4fpl6b83rocnn9f32tsujitalji47hc1mcs7aqs8orsu10rul45r28i0a9/family1/fae569bc02ae4f27b64b648ffd6d17b3 2023-11-27 14:55:22,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:22,787 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:22,789 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120e9359edd22fc4f76a3c790a330456fd2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120e9359edd22fc4f76a3c790a330456fd2 2023-11-27 14:55:22,790 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128c0ce32df9b5c4e939d4b23e8a9f0b4b2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128c0ce32df9b5c4e939d4b23e8a9f0b4b2 2023-11-27 14:55:22,790 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-11-27 14:55:22,790 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-27 14:55:22,791 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:22,791 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:22,791 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:22,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:22,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:22,792 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-27 14:55:22,794 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 900 msec 2023-11-27 14:55:22,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-27 14:55:22,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:23,252 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-11-27 14:55:23,253 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-27 14:55:23,253 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-11-27 14:55:23,255 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-11-27 14:55:23,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, REOPEN/MOVE}] 2023-11-27 14:55:23,258 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, REOPEN/MOVE 2023-11-27 14:55:23,259 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:23,259 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096923259"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096923259"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096923259"}]},"ts":"1701096923259"} 2023-11-27 14:55:23,260 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:23,412 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:23,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:23,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 39f113e3442d78447a6f24b4b73006c8, disabling compactions & flushes 2023-11-27 14:55:23,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:23,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:23,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. after waiting 0 ms 2023-11-27 14:55:23,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:23,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-11-27 14:55:23,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:23,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:23,421 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: 39f113e3442d78447a6f24b4b73006c8 to self. 2023-11-27 14:55:23,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:23,423 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=CLOSED 2023-11-27 14:55:23,423 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096923423"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096923423"}]},"ts":"1701096923423"} 2023-11-27 14:55:23,425 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=52, resume processing ppid=51 2023-11-27 14:55:23,425 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360 in 164 msec 2023-11-27 14:55:23,425 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=true 2023-11-27 14:55:23,576 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:23,576 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096923576"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096923576"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096923576"}]},"ts":"1701096923576"} 2023-11-27 14:55:23,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:23,729 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:23,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:23,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 39f113e3442d78447a6f24b4b73006c8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:55:23,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:23,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:23,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:23,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:23,735 INFO [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:23,736 DEBUG [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:23,736 DEBUG [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1 2023-11-27 14:55:23,736 INFO [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39f113e3442d78447a6f24b4b73006c8 columnFamilyName family1 2023-11-27 14:55:23,742 DEBUG [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/5854fac4e1884090a4abcbc7c04d7444 2023-11-27 14:55:23,745 DEBUG [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/family1/94739b77e35440599c0891206dc25c92_SeqId_53_ 2023-11-27 14:55:23,745 INFO [StoreOpener-39f113e3442d78447a6f24b4b73006c8-1] regionserver.HStore(310): Store=39f113e3442d78447a6f24b4b73006c8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:23,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:23,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:23,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:55:23,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 39f113e3442d78447a6f24b4b73006c8; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65692455, jitterRate=-0.021106138825416565}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:55:23,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:55:23,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8., pid=53, masterSystemTime=1701096923729 2023-11-27 14:55:23,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:23,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:55:23,755 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=39f113e3442d78447a6f24b4b73006c8, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:23,755 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1701096923754"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096923754"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096923754"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096923754"}]},"ts":"1701096923754"} 2023-11-27 14:55:23,757 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=53, resume processing ppid=51 2023-11-27 14:55:23,758 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 39f113e3442d78447a6f24b4b73006c8, server=jenkins-hbase4.apache.org,42669,1701096850360 in 178 msec 2023-11-27 14:55:23,759 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=51, resume processing ppid=50 2023-11-27 14:55:23,759 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39f113e3442d78447a6f24b4b73006c8, REOPEN/MOVE in 501 msec 2023-11-27 14:55:23,761 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=50, resume processing ppid=49 2023-11-27 14:55:23,761 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 504 msec 2023-11-27 14:55:23,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 920 msec 2023-11-27 14:55:23,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-11-27 14:55:23,764 INFO [Listener at localhost/37439] compactions.TestMobCompactor(278): alter status finished 2023-11-27 14:55:23,765 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-27 14:55:23,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-27 14:55:23,767 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-27 14:55:23,767 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097523767 2023-11-27 14:55:23,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:23,768 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:55:23,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-27 14:55:23,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:23,770 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x28a7c4e9 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:55:23,775 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@28015a1f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:55:23,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:23,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:23,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:23,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:23,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:23,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:23,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:23,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-27 14:55:23,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-27 14:55:23,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:23,778 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-27 14:55:23,779 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 12 msec 2023-11-27 14:55:23,827 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=250 (was 263), OpenFileDescriptor=666 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=245 (was 286), ProcessCount=168 (was 168), AvailableMemoryMB=8233 (was 8243) 2023-11-27 14:55:23,839 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=250, OpenFileDescriptor=666, MaxFileDescriptor=60000, SystemLoadAverage=245, ProcessCount=168, AvailableMemoryMB=8232 2023-11-27 14:55:23,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-27 14:55:23,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:23,843 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:55:23,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-11-27 14:55:23,844 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:55:23,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-27 14:55:23,845 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:23,846 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9 empty. 2023-11-27 14:55:23,846 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:23,846 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-11-27 14:55:23,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-27 14:55:24,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-27 14:55:24,254 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-11-27 14:55:24,255 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 36cece40383490f5f6bb34f20558c5b9, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:55:24,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-27 14:55:24,661 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:24,661 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 36cece40383490f5f6bb34f20558c5b9, disabling compactions & flushes 2023-11-27 14:55:24,661 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:24,661 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:24,661 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. after waiting 0 ms 2023-11-27 14:55:24,661 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:24,662 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:24,662 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:24,665 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:55:24,666 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096924666"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096924666"}]},"ts":"1701096924666"} 2023-11-27 14:55:24,667 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-27 14:55:24,668 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:55:24,668 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096924668"}]},"ts":"1701096924668"} 2023-11-27 14:55:24,669 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-11-27 14:55:24,672 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, ASSIGN}] 2023-11-27 14:55:24,673 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, ASSIGN 2023-11-27 14:55:24,674 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:55:24,825 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:24,825 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096924825"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096924825"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096924825"}]},"ts":"1701096924825"} 2023-11-27 14:55:24,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:24,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-27 14:55:24,978 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:24,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:24,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 36cece40383490f5f6bb34f20558c5b9, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:55:24,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:24,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:24,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:24,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:24,983 INFO [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:24,985 DEBUG [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:24,985 DEBUG [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:24,985 INFO [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36cece40383490f5f6bb34f20558c5b9 columnFamilyName family1 2023-11-27 14:55:24,987 INFO [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] regionserver.HStore(310): Store=36cece40383490f5f6bb34f20558c5b9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:24,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:24,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:24,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:24,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:55:24,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 36cece40383490f5f6bb34f20558c5b9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73978323, jitterRate=0.10236291587352753}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:55:24,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:24,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9., pid=57, masterSystemTime=1701096924978 2023-11-27 14:55:24,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:24,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:24,996 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:24,997 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096924996"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096924996"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096924996"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096924996"}]},"ts":"1701096924996"} 2023-11-27 14:55:24,999 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=57, resume processing ppid=56 2023-11-27 14:55:24,999 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360 in 171 msec 2023-11-27 14:55:25,001 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=56, resume processing ppid=55 2023-11-27 14:55:25,001 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, ASSIGN in 327 msec 2023-11-27 14:55:25,001 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:55:25,001 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096925001"}]},"ts":"1701096925001"} 2023-11-27 14:55:25,002 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-11-27 14:55:25,004 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:55:25,005 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1630 sec 2023-11-27 14:55:25,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-27 14:55:25,950 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-11-27 14:55:25,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:25,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:25,977 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:25,977 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:25,977 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:25,977 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:25,978 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:25,978 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:25,981 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:25,981 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:25,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:25,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:25,981 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:25,981 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:25,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:25,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:25,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:25,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:25,982 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:25,982 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:25,982 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:25,982 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:25,982 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:25,983 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:25,983 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:25,983 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:25,983 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:25,984 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-27 14:55:26,001 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d4f71690648d4fe399d5d5902b019a93 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130d4f71690648d4fe399d5d5902b019a93 2023-11-27 14:55:26,007 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/e54626064d9a40388dede7aaaa299776 2023-11-27 14:55:26,011 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/e54626064d9a40388dede7aaaa299776 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e54626064d9a40388dede7aaaa299776 2023-11-27 14:55:26,015 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e54626064d9a40388dede7aaaa299776, entries=1000, sequenceid=5, filesize=190.5 K 2023-11-27 14:55:26,016 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 32ms, sequenceid=5, compaction requested=false 2023-11-27 14:55:26,016 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:26,017 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:26,017 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:26,017 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:26,017 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:26,017 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,017 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:26,017 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:26,019 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,019 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:26,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:26,019 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,019 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:26,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:26,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:26,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:26,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:26,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@309c7443[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:26,021 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:26,021 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,022 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,022 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:26,022 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:26,022 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:26,022 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,023 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:26,024 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,024 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:26,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:26,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:26,024 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:26,024 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:26,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:26,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:26,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:26,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:26,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:26,027 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:26,027 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:26,027 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:26,027 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,027 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:26,037 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,037 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,037 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:26,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,038 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:26,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:26,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,039 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:26,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,041 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,042 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,042 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,042 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,042 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,042 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,042 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,042 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:26,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-27 14:55:26,042 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:26,042 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:26,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:26,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:26,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:26,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:26,042 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:26,043 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:26,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:26,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:26,143 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:26,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:26,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:26,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:26,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,150 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:26,150 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:26,150 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:26,150 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:26,150 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,150 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,152 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,152 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:26,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:26,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:26,152 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,152 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:26,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,153 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:26,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,153 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,153 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-27 14:55:26,153 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:26,154 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:26,154 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:26,154 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:26,154 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-27 14:55:26,568 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511287ff07f11c49a47139101a2c019ad5d24 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511287ff07f11c49a47139101a2c019ad5d24 2023-11-27 14:55:26,975 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/ea49f363367e4ae6a5d9f5791b453fd8 2023-11-27 14:55:26,979 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/ea49f363367e4ae6a5d9f5791b453fd8 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/ea49f363367e4ae6a5d9f5791b453fd8 2023-11-27 14:55:26,983 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/ea49f363367e4ae6a5d9f5791b453fd8, entries=1, sequenceid=9, filesize=5.0 K 2023-11-27 14:55:26,984 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 830ms, sequenceid=9, compaction requested=false 2023-11-27 14:55:26,984 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:26,984 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:26,984 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:26,984 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:26,984 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,984 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:26,984 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:26,986 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,986 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:26,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:26,986 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,986 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:26,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:26,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:26,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:26,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:26,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cdc0644[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:26,987 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:26,987 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,989 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,989 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:26,989 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:26,989 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:26,990 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,990 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:26,991 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:26,991 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,991 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:26,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:26,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:26,991 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:26,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:26,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:26,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:26,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:26,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:26,993 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:26,993 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:26,993 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:26,993 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,993 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:26,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,995 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:26,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:26,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:26,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:26,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:26,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:26,995 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:26,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,997 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:26,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:26,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:26,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,000 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,000 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:27,000 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:27,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:27,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:27,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-27 14:55:27,000 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:27,000 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,000 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:27,001 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,001 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:27,001 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:27,001 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:27,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:27,001 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,001 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,087 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:55:27,101 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:27,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:27,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:27,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:27,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,108 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:27,108 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:27,108 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:27,108 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:27,109 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,109 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,112 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:27,112 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:27,112 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,112 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:27,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:27,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,113 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-27 14:55:27,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:27,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:27,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:27,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:27,114 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:27,114 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:27,114 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:27,114 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:27,538 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120162f06c1dfeb441c5ab7c258f21b85b6b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120162f06c1dfeb441c5ab7c258f21b85b6b 2023-11-27 14:55:27,944 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/29f740d017864514ab0d371cbd018229 2023-11-27 14:55:27,949 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/29f740d017864514ab0d371cbd018229 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/29f740d017864514ab0d371cbd018229 2023-11-27 14:55:27,953 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/29f740d017864514ab0d371cbd018229, entries=1, sequenceid=13, filesize=5.0 K 2023-11-27 14:55:27,954 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 840ms, sequenceid=13, compaction requested=true 2023-11-27 14:55:27,954 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:27,954 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:27,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:27,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:27,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:27,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:27,956 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,956 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:27,957 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,957 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:27,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:27,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:27,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:27,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:27,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63df24e7[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:27,959 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:27,959 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,960 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,960 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:27,960 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:27,960 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:27,960 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,960 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:27,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:27,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:27,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:27,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:27,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:27,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:27,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:27,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:27,964 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:27,964 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:27,965 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:27,965 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,965 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:27,967 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,967 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:27,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:27,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:27,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:27,968 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:27,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:27,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:27,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,969 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:27,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,974 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,974 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,974 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,974 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:27,974 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:27,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:27,974 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:27,974 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:27,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:27,974 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:27,974 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,975 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:27,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:27,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:27,975 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:27,975 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:28,075 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:28,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:28,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:28,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:28,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,081 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:28,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:28,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:28,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:28,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,082 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,086 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:28,086 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:28,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:28,086 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,086 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:28,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:28,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,088 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-27 14:55:28,088 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,088 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:28,088 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:28,088 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:28,088 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:28,088 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:28,088 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:28,088 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:28,088 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:28,502 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205524d9f1a743f4804932dd93021cfaa7f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205524d9f1a743f4804932dd93021cfaa7f 2023-11-27 14:55:28,908 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/d0a130f7373047289d6b584f504ee00a 2023-11-27 14:55:28,913 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/d0a130f7373047289d6b584f504ee00a as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/d0a130f7373047289d6b584f504ee00a 2023-11-27 14:55:28,918 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/d0a130f7373047289d6b584f504ee00a, entries=1, sequenceid=17, filesize=5.0 K 2023-11-27 14:55:28,918 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 830ms, sequenceid=17, compaction requested=true 2023-11-27 14:55:28,919 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:28,919 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:28,919 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:28,919 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:28,919 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,919 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:28,919 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:28,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,921 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:28,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:28,921 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:28,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:28,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:28,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:28,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:28,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42b0d985[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:28,923 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:28,923 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,924 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:28,924 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,924 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:28,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:28,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:28,926 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:28,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:28,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:28,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:28,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:28,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:28,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:28,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:28,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:28,928 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:28,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:28,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:28,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,928 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:28,930 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,930 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:28,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:28,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,930 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:28,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:28,930 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:28,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:28,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:28,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,932 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:28,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:28,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,935 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:28,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:28,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:28,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:28,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:28,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:28,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:28,936 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,936 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,936 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:28,936 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:28,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:28,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:28,936 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:29,036 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:29,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:29,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:29,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:29,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,043 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:29,043 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:29,043 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:29,043 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:29,043 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,044 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,045 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:29,045 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:29,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:29,045 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,045 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:29,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,045 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:29,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,046 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,046 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-27 14:55:29,046 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:29,046 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:29,046 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:29,046 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:29,462 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122865f3eac4b14e4735b25f5d98d56774d1 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122865f3eac4b14e4735b25f5d98d56774d1 2023-11-27 14:55:29,468 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/c057d801c4b74cd8a2cc84da188ac7df 2023-11-27 14:55:29,472 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/c057d801c4b74cd8a2cc84da188ac7df as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/c057d801c4b74cd8a2cc84da188ac7df 2023-11-27 14:55:29,476 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/c057d801c4b74cd8a2cc84da188ac7df, entries=1, sequenceid=21, filesize=5.0 K 2023-11-27 14:55:29,477 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 431ms, sequenceid=21, compaction requested=true 2023-11-27 14:55:29,477 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:29,477 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:29,477 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:29,477 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:29,478 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,478 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:29,478 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:29,479 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,479 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:29,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:29,480 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,480 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:29,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:29,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:29,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:29,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:29,481 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@115d83fc[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:29,481 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:29,482 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,483 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,483 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:29,483 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:29,483 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,483 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:29,483 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:29,485 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,485 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:29,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:29,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:29,485 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:29,485 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:29,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:29,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:29,487 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:29,487 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:29,487 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:29,487 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:29,487 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,487 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:29,489 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,489 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:29,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:29,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,489 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:29,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:29,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:29,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:29,489 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:29,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,491 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:29,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:29,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:29,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:29,493 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:29,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:29,493 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:29,494 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:29,594 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:29,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:29,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:29,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:29,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,600 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:29,600 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:29,600 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:29,600 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:29,600 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,600 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,601 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,601 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:29,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:29,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:29,602 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:29,602 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:29,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:29,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,603 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-27 14:55:29,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:29,603 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:29,603 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:29,603 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:29,603 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:29,603 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:29,603 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:29,604 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:29,604 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:29,629 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-27 14:55:30,016 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512314c6931aa9c1a4b34bd996f7042a54be7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512314c6931aa9c1a4b34bd996f7042a54be7 2023-11-27 14:55:30,422 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/3cafa5d03318436b9d3237189d6c6444 2023-11-27 14:55:30,427 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/3cafa5d03318436b9d3237189d6c6444 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/3cafa5d03318436b9d3237189d6c6444 2023-11-27 14:55:30,431 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/3cafa5d03318436b9d3237189d6c6444, entries=1, sequenceid=25, filesize=5.0 K 2023-11-27 14:55:30,432 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 828ms, sequenceid=25, compaction requested=true 2023-11-27 14:55:30,432 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:30,432 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:30,432 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:30,432 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:30,432 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,432 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:30,432 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:30,434 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,434 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:30,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:30,435 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,435 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:30,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:30,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:30,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:30,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:30,436 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:30,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5dff59b5[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:30,436 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,437 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,437 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:30,437 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:30,437 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:30,438 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,438 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:30,439 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,439 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:30,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:30,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:30,439 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:30,439 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:30,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:30,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:30,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:30,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:30,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:30,441 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:30,441 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:30,441 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:30,441 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,441 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:30,443 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,443 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:30,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:30,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,443 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:30,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:30,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:30,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:30,449 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:30,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,450 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:30,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,453 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,453 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,453 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:30,453 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:30,453 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:30,453 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:30,453 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:30,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:30,453 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:30,454 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,454 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:30,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:30,454 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:30,454 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:30,554 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:30,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:30,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:30,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:30,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,561 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:30,561 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:30,561 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:30,561 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:30,562 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,562 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,563 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:30,563 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:30,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:30,563 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:30,563 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:30,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:30,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,564 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-27 14:55:30,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:30,565 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:30,565 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:30,565 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:30,565 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:30,565 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:30,565 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:30,565 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:30,565 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:30,977 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101aff58de3366641f899e827f77580d206 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101aff58de3366641f899e827f77580d206 2023-11-27 14:55:31,383 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/2390cffc8def494187aaa9c8643ea07c 2023-11-27 14:55:31,388 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/2390cffc8def494187aaa9c8643ea07c as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/2390cffc8def494187aaa9c8643ea07c 2023-11-27 14:55:31,392 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/2390cffc8def494187aaa9c8643ea07c, entries=1, sequenceid=29, filesize=5.0 K 2023-11-27 14:55:31,393 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 828ms, sequenceid=29, compaction requested=true 2023-11-27 14:55:31,393 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:31,393 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:31,393 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:31,393 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:31,393 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,393 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:31,393 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:31,395 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,395 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:31,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:31,395 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,395 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:31,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:31,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@479da6f1[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:31,397 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:31,397 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,398 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,398 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:31,398 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:31,398 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:31,398 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,398 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:31,400 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:31,400 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,400 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:31,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:31,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:31,400 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:31,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:31,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:31,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:31,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:31,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:31,403 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:31,403 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:31,403 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:31,403 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,403 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:31,404 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,404 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:31,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:31,405 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:31,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:31,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:31,405 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:31,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:31,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,406 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:31,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,409 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:31,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:31,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:31,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:31,409 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:31,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:31,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:31,509 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:31,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:31,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:31,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:31,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,517 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:31,517 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:31,517 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:31,517 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:31,517 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,517 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,519 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:31,519 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:31,519 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,519 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,520 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-27 14:55:31,520 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,520 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:31,520 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:31,520 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:31,524 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:31,524 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:31,524 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:31,524 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:31,524 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:31,544 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010383d047228b4c496994b08271d9ee8f07 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010383d047228b4c496994b08271d9ee8f07 2023-11-27 14:55:31,954 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/48ec551e386746a1bd9b21b07860c1b9 2023-11-27 14:55:31,959 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/48ec551e386746a1bd9b21b07860c1b9 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/48ec551e386746a1bd9b21b07860c1b9 2023-11-27 14:55:31,964 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/48ec551e386746a1bd9b21b07860c1b9, entries=1, sequenceid=33, filesize=5.0 K 2023-11-27 14:55:31,965 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 441ms, sequenceid=33, compaction requested=true 2023-11-27 14:55:31,965 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:31,965 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:31,965 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:31,968 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:31,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,968 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:31,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:31,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:31,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:31,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:31,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3252376[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:31,970 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:31,970 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,971 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,971 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:31,971 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,971 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:31,971 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:31,971 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:31,975 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,975 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:31,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:31,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:31,975 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:31,975 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:31,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:31,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:31,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:31,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:31,978 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:31,978 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:31,978 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:31,978 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:31,978 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,978 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:31,980 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,980 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,981 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:31,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:31,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:31,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:31,981 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:31,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:31,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:31,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,982 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:31,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:31,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:31,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:31,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-27 14:55:31,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:31,985 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:31,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:31,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:31,985 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:32,086 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:32,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:32,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:32,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:32,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,092 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:32,092 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:32,092 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:32,092 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:32,093 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,093 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,094 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:32,094 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:32,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:32,095 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,095 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:32,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:32,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,095 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-27 14:55:32,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,096 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:32,096 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:32,096 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:32,096 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:32,096 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:32,096 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:32,096 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:32,096 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:32,515 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907a0c09d3522644bce96eb58377751fb84 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907a0c09d3522644bce96eb58377751fb84 2023-11-27 14:55:32,921 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/44b1a52bcda5452dbeaa08e7c83221fe 2023-11-27 14:55:32,926 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/44b1a52bcda5452dbeaa08e7c83221fe as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/44b1a52bcda5452dbeaa08e7c83221fe 2023-11-27 14:55:32,931 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/44b1a52bcda5452dbeaa08e7c83221fe, entries=1, sequenceid=37, filesize=5.0 K 2023-11-27 14:55:32,932 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 836ms, sequenceid=37, compaction requested=true 2023-11-27 14:55:32,932 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:32,932 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:32,933 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:32,933 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:32,933 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,933 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:32,933 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:32,934 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,934 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:32,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:32,935 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,935 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:32,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:32,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:32,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:32,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:32,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e1a128d[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:32,937 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:32,937 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,939 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,939 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:32,939 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:32,939 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:32,939 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,939 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:32,940 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,941 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:32,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:32,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:32,941 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:32,941 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:32,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:32,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:32,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:32,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:32,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:32,944 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:32,944 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:32,944 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:32,944 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,944 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:32,946 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,946 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,946 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:32,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:32,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:32,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,946 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:32,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:32,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:32,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:32,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,948 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:32,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:32,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:32,951 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:32,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:32,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:32,951 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-27 14:55:32,952 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:32,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:32,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:33,052 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:33,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:33,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:33,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:33,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,059 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:33,060 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:33,060 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-27 14:55:33,060 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-27 14:55:33,060 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,060 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,063 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:33,063 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:33,063 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,063 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,064 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,064 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,065 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-27 14:55:33,065 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,065 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-27 14:55:33,065 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:33,065 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-27 14:55:33,065 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:55:33,065 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:33,065 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:33,065 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. started... 2023-11-27 14:55:33,065 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 36cece40383490f5f6bb34f20558c5b9 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:55:33,484 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120b115cc6c2f3e4f88883bbfc93da90bf0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120b115cc6c2f3e4f88883bbfc93da90bf0 2023-11-27 14:55:33,890 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/a66de98e61664aedb61e186c82b2da87 2023-11-27 14:55:33,914 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:55:33,914 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x489a311b to 127.0.0.1:61098 2023-11-27 14:55:33,914 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:55:33,914 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28a7c4e9 to 127.0.0.1:61098 2023-11-27 14:55:33,914 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:55:33,915 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:55:33,915 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x67971a8b to 127.0.0.1:61098 2023-11-27 14:55:33,915 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:55:33,915 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:55:33,915 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x381c1de3 to 127.0.0.1:61098 2023-11-27 14:55:33,915 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:55:33,916 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:55:33,916 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x11129896 to 127.0.0.1:61098 2023-11-27 14:55:33,916 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:55:33,918 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/a66de98e61664aedb61e186c82b2da87 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/a66de98e61664aedb61e186c82b2da87 2023-11-27 14:55:33,923 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/a66de98e61664aedb61e186c82b2da87, entries=1, sequenceid=41, filesize=5.0 K 2023-11-27 14:55:33,924 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 36cece40383490f5f6bb34f20558c5b9 in 859ms, sequenceid=41, compaction requested=true 2023-11-27 14:55:33,924 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:33,924 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:33,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:55:33,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:55:33,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-27 14:55:33,924 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-27 14:55:33,925 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,925 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:33,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:33,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,926 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:33,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:33,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:33,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-27 14:55:33,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@590fce0a[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:33,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-27 14:55:33,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,935 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-27 14:55:33,935 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-27 14:55:33,935 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,935 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:33,935 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:33,936 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-27 14:55:33,937 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:33,936 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,937 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-27 14:55:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:33,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:33,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:33,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-27 14:55:33,939 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-27 14:55:33,939 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:33,939 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:33,939 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,939 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:33,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,944 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:33,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:33,944 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:33,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,946 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:33,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,948 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,948 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:33,948 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,948 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:33,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:33,948 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:33,948 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-27 14:55:33,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:33,948 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:33,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:33,949 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,949 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,949 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:33,949 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:33,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:33,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:33,949 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:34,049 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:34,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:34,051 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:34,053 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-27 14:55:34,054 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-27 14:55:34,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:34,054 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097534054 2023-11-27 14:55:34,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:55:34,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-27 14:55:34,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:34,058 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x3345192b to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:55:34,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4a38e985, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:55:34,063 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:34,899 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103903ce4e152c94e7a8a7c40d14c7d3fd1 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103903ce4e152c94e7a8a7c40d14c7d3fd1 2023-11-27 14:55:34,900 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:34,913 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:55:34,914 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38922, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:55:34,921 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:55:34,922 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40844, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:55:34,930 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b40fbe0a664748ccbd619d4fc83b4526 first=Optional[r3] last=Optional[r6] 2023-11-27 14:55:34,931 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b40fbe0a664748ccbd619d4fc83b4526}] 2023-11-27 14:55:34,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b40fbe0a664748ccbd619d4fc83b4526 for inclusion in 36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:34,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-27 14:55:34,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:55:34,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:34,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b40fbe0a664748ccbd619d4fc83b4526 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8a01tlpvllimir5fgi2qsu810q5r6i559gdppvkarrbfhlvjer3ubeebhrvdonpo/family1/b40fbe0a664748ccbd619d4fc83b4526 2023-11-27 14:55:34,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8a01tlpvllimir5fgi2qsu810q5r6i559gdppvkarrbfhlvjer3ubeebhrvdonpo/family1/b40fbe0a664748ccbd619d4fc83b4526 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/cc27656aef314379b10e5c6acb259987_SeqId_44_ 2023-11-27 14:55:34,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8a01tlpvllimir5fgi2qsu810q5r6i559gdppvkarrbfhlvjer3ubeebhrvdonpo/family1/b40fbe0a664748ccbd619d4fc83b4526 into 36cece40383490f5f6bb34f20558c5b9/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/cc27656aef314379b10e5c6acb259987_SeqId_44_ - updating store file list. 2023-11-27 14:55:34,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/cc27656aef314379b10e5c6acb259987_SeqId_44_ into 36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:34,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8a01tlpvllimir5fgi2qsu810q5r6i559gdppvkarrbfhlvjer3ubeebhrvdonpo/family1/b40fbe0a664748ccbd619d4fc83b4526 into 36cece40383490f5f6bb34f20558c5b9/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/cc27656aef314379b10e5c6acb259987_SeqId_44_) 2023-11-27 14:55:34,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__8a01tlpvllimir5fgi2qsu810q5r6i559gdppvkarrbfhlvjer3ubeebhrvdonpo/family1/b40fbe0a664748ccbd619d4fc83b4526 2023-11-27 14:55:34,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-27 14:55:34,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-27 14:55:34,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): 36cece40383490f5f6bb34f20558c5b9/family1 is initiating minor compaction 2023-11-27 14:55:34,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@30baa311; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:34,950 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 36cece40383490f5f6bb34f20558c5b9/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:34,950 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/ea49f363367e4ae6a5d9f5791b453fd8, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/29f740d017864514ab0d371cbd018229, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/d0a130f7373047289d6b584f504ee00a, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/c057d801c4b74cd8a2cc84da188ac7df, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/3cafa5d03318436b9d3237189d6c6444, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/2390cffc8def494187aaa9c8643ea07c, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/48ec551e386746a1bd9b21b07860c1b9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/44b1a52bcda5452dbeaa08e7c83221fe, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/a66de98e61664aedb61e186c82b2da87, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/cc27656aef314379b10e5c6acb259987_SeqId_44_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp, totalSize=50.3 K 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting ea49f363367e4ae6a5d9f5791b453fd8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 29f740d017864514ab0d371cbd018229, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting d0a130f7373047289d6b584f504ee00a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting c057d801c4b74cd8a2cc84da188ac7df, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 3cafa5d03318436b9d3237189d6c6444, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 2390cffc8def494187aaa9c8643ea07c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 48ec551e386746a1bd9b21b07860c1b9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 44b1a52bcda5452dbeaa08e7c83221fe, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting a66de98e61664aedb61e186c82b2da87, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-27 14:55:34,951 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting cc27656aef314379b10e5c6acb259987_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-27 14:55:34,953 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:34,954 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:34,956 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122865f3eac4b14e4735b25f5d98d56774d1 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122865f3eac4b14e4735b25f5d98d56774d1 2023-11-27 14:55:34,959 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512314c6931aa9c1a4b34bd996f7042a54be7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512314c6931aa9c1a4b34bd996f7042a54be7 2023-11-27 14:55:34,961 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101aff58de3366641f899e827f77580d206 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101aff58de3366641f899e827f77580d206 2023-11-27 14:55:34,963 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010383d047228b4c496994b08271d9ee8f07 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010383d047228b4c496994b08271d9ee8f07 2023-11-27 14:55:34,963 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-11-27 14:55:34,963 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-27 14:55:34,964 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:34,964 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:34,964 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:34,964 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:34,964 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:34,964 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:34,964 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:34,965 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:34,980 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 36cece40383490f5f6bb34f20558c5b9#family1#compaction#71 average throughput is 1.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:55:35,804 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/5e17a0bdd4c6473a813910a2d065ff1f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5e17a0bdd4c6473a813910a2d065ff1f 2023-11-27 14:55:35,809 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 36cece40383490f5f6bb34f20558c5b9/family1 of 36cece40383490f5f6bb34f20558c5b9 into 5e17a0bdd4c6473a813910a2d065ff1f(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:55:35,810 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:35,810 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1701096934950; duration=0sec 2023-11-27 14:55:35,810 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:35,820 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054be3bf4314e940b796c265cc1c0db33b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512054be3bf4314e940b796c265cc1c0db33b 2023-11-27 14:55:35,821 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:35,852 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e5e6201b635842c1949ba73808e620de first=Optional[r00] last=Optional[r2] 2023-11-27 14:55:35,853 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e5e6201b635842c1949ba73808e620de}] 2023-11-27 14:55:35,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e5e6201b635842c1949ba73808e620de for inclusion in 36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:35,869 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/ea49f363367e4ae6a5d9f5791b453fd8, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/29f740d017864514ab0d371cbd018229, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/d0a130f7373047289d6b584f504ee00a, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/c057d801c4b74cd8a2cc84da188ac7df, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/3cafa5d03318436b9d3237189d6c6444, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/2390cffc8def494187aaa9c8643ea07c, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/48ec551e386746a1bd9b21b07860c1b9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/44b1a52bcda5452dbeaa08e7c83221fe, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/a66de98e61664aedb61e186c82b2da87, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/cc27656aef314379b10e5c6acb259987_SeqId_44_] to archive 2023-11-27 14:55:35,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-27 14:55:35,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:55:35,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:35,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:35,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e5e6201b635842c1949ba73808e620de to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ckcpv2kofq2h5df0barcqtnm6fk9vrhharlk66ms7keftf06juk5qgt91j46789n/family1/e5e6201b635842c1949ba73808e620de 2023-11-27 14:55:35,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/ea49f363367e4ae6a5d9f5791b453fd8 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/ea49f363367e4ae6a5d9f5791b453fd8 2023-11-27 14:55:35,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/29f740d017864514ab0d371cbd018229 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/29f740d017864514ab0d371cbd018229 2023-11-27 14:55:35,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ckcpv2kofq2h5df0barcqtnm6fk9vrhharlk66ms7keftf06juk5qgt91j46789n/family1/e5e6201b635842c1949ba73808e620de as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e1faaffed8c542fbbea3a2933c0221e3_SeqId_47_ 2023-11-27 14:55:35,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ckcpv2kofq2h5df0barcqtnm6fk9vrhharlk66ms7keftf06juk5qgt91j46789n/family1/e5e6201b635842c1949ba73808e620de into 36cece40383490f5f6bb34f20558c5b9/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e1faaffed8c542fbbea3a2933c0221e3_SeqId_47_ - updating store file list. 2023-11-27 14:55:35,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/d0a130f7373047289d6b584f504ee00a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/d0a130f7373047289d6b584f504ee00a 2023-11-27 14:55:35,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/c057d801c4b74cd8a2cc84da188ac7df to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/c057d801c4b74cd8a2cc84da188ac7df 2023-11-27 14:55:35,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/3cafa5d03318436b9d3237189d6c6444 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/3cafa5d03318436b9d3237189d6c6444 2023-11-27 14:55:35,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e1faaffed8c542fbbea3a2933c0221e3_SeqId_47_ into 36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:35,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ckcpv2kofq2h5df0barcqtnm6fk9vrhharlk66ms7keftf06juk5qgt91j46789n/family1/e5e6201b635842c1949ba73808e620de into 36cece40383490f5f6bb34f20558c5b9/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e1faaffed8c542fbbea3a2933c0221e3_SeqId_47_) 2023-11-27 14:55:35,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ckcpv2kofq2h5df0barcqtnm6fk9vrhharlk66ms7keftf06juk5qgt91j46789n/family1/e5e6201b635842c1949ba73808e620de 2023-11-27 14:55:35,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/2390cffc8def494187aaa9c8643ea07c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/2390cffc8def494187aaa9c8643ea07c 2023-11-27 14:55:35,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-27 14:55:35,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-27 14:55:35,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): 36cece40383490f5f6bb34f20558c5b9/family1 is initiating minor compaction (all files) 2023-11-27 14:55:35,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@538b67a0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:35,880 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 36cece40383490f5f6bb34f20558c5b9/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:35,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/48ec551e386746a1bd9b21b07860c1b9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/48ec551e386746a1bd9b21b07860c1b9 2023-11-27 14:55:35,880 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e54626064d9a40388dede7aaaa299776, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5e17a0bdd4c6473a813910a2d065ff1f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e1faaffed8c542fbbea3a2933c0221e3_SeqId_47_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp, totalSize=388.1 K 2023-11-27 14:55:35,881 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting e54626064d9a40388dede7aaaa299776, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878813266 2023-11-27 14:55:35,881 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 5e17a0bdd4c6473a813910a2d065ff1f, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621213266 2023-11-27 14:55:35,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/44b1a52bcda5452dbeaa08e7c83221fe to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/44b1a52bcda5452dbeaa08e7c83221fe 2023-11-27 14:55:35,882 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:35,882 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting e1faaffed8c542fbbea3a2933c0221e3_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878813266 2023-11-27 14:55:35,883 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:35,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/a66de98e61664aedb61e186c82b2da87 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/a66de98e61664aedb61e186c82b2da87 2023-11-27 14:55:35,884 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130d4f71690648d4fe399d5d5902b019a93 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130d4f71690648d4fe399d5d5902b019a93 2023-11-27 14:55:35,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/cc27656aef314379b10e5c6acb259987_SeqId_44_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/cc27656aef314379b10e5c6acb259987_SeqId_44_ 2023-11-27 14:55:35,885 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120162f06c1dfeb441c5ab7c258f21b85b6b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120162f06c1dfeb441c5ab7c258f21b85b6b 2023-11-27 14:55:35,887 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205524d9f1a743f4804932dd93021cfaa7f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205524d9f1a743f4804932dd93021cfaa7f 2023-11-27 14:55:35,887 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-11-27 14:55:35,887 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-27 14:55:35,887 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:35,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:35,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:35,888 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-27 14:55:35,889 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8360 sec 2023-11-27 14:55:35,900 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 36cece40383490f5f6bb34f20558c5b9#family1#compaction#72 average throughput is 22.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:55:35,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-27 14:55:35,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:36,382 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-27 14:55:36,383 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-11-27 14:55:36,384 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-27 14:55:36,385 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-11-27 14:55:36,388 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, REOPEN/MOVE}] 2023-11-27 14:55:36,389 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, REOPEN/MOVE 2023-11-27 14:55:36,389 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:36,390 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096936389"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096936389"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096936389"}]},"ts":"1701096936389"} 2023-11-27 14:55:36,391 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:36,542 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:36,543 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:36,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 36cece40383490f5f6bb34f20558c5b9, disabling compactions & flushes 2023-11-27 14:55:36,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:37,119 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/.tmp/family1/1f34f4f58c0d4628855403d37906c3fa as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/1f34f4f58c0d4628855403d37906c3fa 2023-11-27 14:55:37,124 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 36cece40383490f5f6bb34f20558c5b9/family1 of 36cece40383490f5f6bb34f20558c5b9 into 1f34f4f58c0d4628855403d37906c3fa(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:55:37,124 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:37,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:37,124 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1701096935880; duration=1sec 2023-11-27 14:55:37,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:37,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. after waiting 0 ms 2023-11-27 14:55:37,124 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:37,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:37,127 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e54626064d9a40388dede7aaaa299776, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5e17a0bdd4c6473a813910a2d065ff1f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e1faaffed8c542fbbea3a2933c0221e3_SeqId_47_] to archive 2023-11-27 14:55:37,128 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:37,130 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e54626064d9a40388dede7aaaa299776 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e54626064d9a40388dede7aaaa299776 2023-11-27 14:55:37,131 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5e17a0bdd4c6473a813910a2d065ff1f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5e17a0bdd4c6473a813910a2d065ff1f 2023-11-27 14:55:37,132 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e1faaffed8c542fbbea3a2933c0221e3_SeqId_47_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/e1faaffed8c542fbbea3a2933c0221e3_SeqId_47_ 2023-11-27 14:55:37,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-27 14:55:37,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:37,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:37,137 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: 36cece40383490f5f6bb34f20558c5b9 to self. 2023-11-27 14:55:37,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:37,139 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=CLOSED 2023-11-27 14:55:37,139 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096937139"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096937139"}]},"ts":"1701096937139"} 2023-11-27 14:55:37,141 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=62, resume processing ppid=61 2023-11-27 14:55:37,141 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360 in 749 msec 2023-11-27 14:55:37,142 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=true 2023-11-27 14:55:37,292 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:37,292 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096937292"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096937292"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096937292"}]},"ts":"1701096937292"} 2023-11-27 14:55:37,294 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:37,445 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:37,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:37,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 36cece40383490f5f6bb34f20558c5b9, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:55:37,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:37,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:37,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:37,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:37,450 INFO [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:37,451 DEBUG [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:37,451 DEBUG [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:37,452 INFO [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36cece40383490f5f6bb34f20558c5b9 columnFamilyName family1 2023-11-27 14:55:37,456 DEBUG [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/1f34f4f58c0d4628855403d37906c3fa 2023-11-27 14:55:37,456 INFO [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] regionserver.HStore(310): Store=36cece40383490f5f6bb34f20558c5b9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:37,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:37,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:37,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:37,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 36cece40383490f5f6bb34f20558c5b9; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69946398, jitterRate=0.042282551527023315}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:55:37,462 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:37,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9., pid=63, masterSystemTime=1701096937445 2023-11-27 14:55:37,463 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:37,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:37,464 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:37,464 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096937464"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096937464"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096937464"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096937464"}]},"ts":"1701096937464"} 2023-11-27 14:55:37,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=63, resume processing ppid=61 2023-11-27 14:55:37,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360 in 171 msec 2023-11-27 14:55:37,467 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=61, resume processing ppid=60 2023-11-27 14:55:37,467 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, REOPEN/MOVE in 1.0780 sec 2023-11-27 14:55:37,469 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=60, resume processing ppid=59 2023-11-27 14:55:37,469 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0820 sec 2023-11-27 14:55:37,470 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.4990 sec 2023-11-27 14:55:37,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=59 2023-11-27 14:55:37,473 INFO [Listener at localhost/37439] compactions.TestMobCompactor(278): alter status finished 2023-11-27 14:55:37,474 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:37,475 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-27 14:55:37,476 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-27 14:55:37,476 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:37,476 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097537476 2023-11-27 14:55:37,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:55:37,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-27 14:55:37,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:37,479 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x777b5fe7 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:55:37,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c75dfe6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:55:37,488 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:37,488 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:37,488 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:37,488 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:37,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:37,918 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b8e9fcee2e0543869da1d945f47c5214 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128b8e9fcee2e0543869da1d945f47c5214 2023-11-27 14:55:37,919 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:37,932 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:55:37,934 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38926, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:55:37,941 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:55:37,942 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40858, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:55:37,950 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2a8d82ef8eca4070b13231cf126be6d6 first=Optional[r05] last=Optional[r8] 2023-11-27 14:55:37,951 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=51 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2a8d82ef8eca4070b13231cf126be6d6}] 2023-11-27 14:55:37,955 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2a8d82ef8eca4070b13231cf126be6d6 for inclusion in 36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:37,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-27 14:55:37,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:55:37,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:37,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2a8d82ef8eca4070b13231cf126be6d6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3ivt05j8jucanbovf979qtu4at2eh48ev1r9k3vja61mu0l1s720trnou31jc96d/family1/2a8d82ef8eca4070b13231cf126be6d6 2023-11-27 14:55:37,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3ivt05j8jucanbovf979qtu4at2eh48ev1r9k3vja61mu0l1s720trnou31jc96d/family1/2a8d82ef8eca4070b13231cf126be6d6 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5bd682a42b44493aa8e05069071c5ea3_SeqId_53_ 2023-11-27 14:55:37,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3ivt05j8jucanbovf979qtu4at2eh48ev1r9k3vja61mu0l1s720trnou31jc96d/family1/2a8d82ef8eca4070b13231cf126be6d6 into 36cece40383490f5f6bb34f20558c5b9/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5bd682a42b44493aa8e05069071c5ea3_SeqId_53_ - updating store file list. 2023-11-27 14:55:37,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5bd682a42b44493aa8e05069071c5ea3_SeqId_53_ into 36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:37,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3ivt05j8jucanbovf979qtu4at2eh48ev1r9k3vja61mu0l1s720trnou31jc96d/family1/2a8d82ef8eca4070b13231cf126be6d6 into 36cece40383490f5f6bb34f20558c5b9/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5bd682a42b44493aa8e05069071c5ea3_SeqId_53_) 2023-11-27 14:55:37,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3ivt05j8jucanbovf979qtu4at2eh48ev1r9k3vja61mu0l1s720trnou31jc96d/family1/2a8d82ef8eca4070b13231cf126be6d6 2023-11-27 14:55:37,965 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:37,966 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:37,967 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120b115cc6c2f3e4f88883bbfc93da90bf0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120b115cc6c2f3e4f88883bbfc93da90bf0 2023-11-27 14:55:37,968 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511287ff07f11c49a47139101a2c019ad5d24 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511287ff07f11c49a47139101a2c019ad5d24 2023-11-27 14:55:37,968 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-11-27 14:55:37,968 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-27 14:55:37,969 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:37,969 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:37,969 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:37,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:37,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:37,970 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-27 14:55:37,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 496 msec 2023-11-27 14:55:38,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-11-27 14:55:38,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:38,031 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-11-27 14:55:38,032 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-27 14:55:38,032 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-11-27 14:55:38,033 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-11-27 14:55:38,035 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, REOPEN/MOVE}] 2023-11-27 14:55:38,036 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, REOPEN/MOVE 2023-11-27 14:55:38,037 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:38,037 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096938037"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096938037"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096938037"}]},"ts":"1701096938037"} 2023-11-27 14:55:38,038 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:38,189 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:38,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:38,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 36cece40383490f5f6bb34f20558c5b9, disabling compactions & flushes 2023-11-27 14:55:38,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:38,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:38,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. after waiting 0 ms 2023-11-27 14:55:38,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:38,197 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-11-27 14:55:38,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:38,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:38,198 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3512): Not adding moved region record: 36cece40383490f5f6bb34f20558c5b9 to self. 2023-11-27 14:55:38,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:38,199 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=CLOSED 2023-11-27 14:55:38,200 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096938199"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096938199"}]},"ts":"1701096938199"} 2023-11-27 14:55:38,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=68, resume processing ppid=67 2023-11-27 14:55:38,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360 in 163 msec 2023-11-27 14:55:38,202 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=true 2023-11-27 14:55:38,353 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:38,353 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096938353"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096938353"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096938353"}]},"ts":"1701096938353"} 2023-11-27 14:55:38,355 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:38,506 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:38,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:38,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 36cece40383490f5f6bb34f20558c5b9, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:55:38,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:38,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:38,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:38,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:38,510 INFO [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:38,511 DEBUG [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:38,511 DEBUG [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1 2023-11-27 14:55:38,512 INFO [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36cece40383490f5f6bb34f20558c5b9 columnFamilyName family1 2023-11-27 14:55:38,516 DEBUG [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/1f34f4f58c0d4628855403d37906c3fa 2023-11-27 14:55:38,519 DEBUG [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/family1/5bd682a42b44493aa8e05069071c5ea3_SeqId_53_ 2023-11-27 14:55:38,519 INFO [StoreOpener-36cece40383490f5f6bb34f20558c5b9-1] regionserver.HStore(310): Store=36cece40383490f5f6bb34f20558c5b9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:38,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:38,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:38,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:55:38,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 36cece40383490f5f6bb34f20558c5b9; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68450550, jitterRate=0.019992679357528687}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:55:38,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:55:38,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9., pid=69, masterSystemTime=1701096938506 2023-11-27 14:55:38,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:38,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:55:38,526 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=36cece40383490f5f6bb34f20558c5b9, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:38,526 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1701096938525"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096938525"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096938525"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096938525"}]},"ts":"1701096938525"} 2023-11-27 14:55:38,528 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=69, resume processing ppid=67 2023-11-27 14:55:38,528 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 36cece40383490f5f6bb34f20558c5b9, server=jenkins-hbase4.apache.org,42669,1701096850360 in 172 msec 2023-11-27 14:55:38,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=67, resume processing ppid=66 2023-11-27 14:55:38,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=36cece40383490f5f6bb34f20558c5b9, REOPEN/MOVE in 493 msec 2023-11-27 14:55:38,531 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=66, resume processing ppid=65 2023-11-27 14:55:38,531 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 497 msec 2023-11-27 14:55:38,532 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 510 msec 2023-11-27 14:55:38,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=65 2023-11-27 14:55:38,534 INFO [Listener at localhost/37439] compactions.TestMobCompactor(278): alter status finished 2023-11-27 14:55:38,535 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-27 14:55:38,536 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-27 14:55:38,537 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-27 14:55:38,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:38,538 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097538538 2023-11-27 14:55:38,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:55:38,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-27 14:55:38,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:38,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x7cf997cc to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:55:38,544 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4f619df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:55:38,544 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:38,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:38,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:38,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:38,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:38,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:38,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:38,545 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-27 14:55:38,545 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-27 14:55:38,545 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:38,545 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-27 14:55:38,546 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 10 msec 2023-11-27 14:55:38,594 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=243 (was 250), OpenFileDescriptor=645 (was 666), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=236 (was 245), ProcessCount=168 (was 168), AvailableMemoryMB=8182 (was 8232) 2023-11-27 14:55:38,607 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=243, OpenFileDescriptor=645, MaxFileDescriptor=60000, SystemLoadAverage=236, ProcessCount=168, AvailableMemoryMB=8181 2023-11-27 14:55:38,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-27 14:55:38,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-11-27 14:55:38,613 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:55:38,613 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-11-27 14:55:38,613 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:55:38,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-27 14:55:38,615 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:38,615 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a empty. 2023-11-27 14:55:38,615 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:38,615 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-11-27 14:55:38,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-27 14:55:38,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-27 14:55:39,023 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-11-27 14:55:39,024 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => f29516ec1eaf69d9b14953c78471722a, NAME => 'testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:55:39,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-27 14:55:39,430 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:39,430 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing f29516ec1eaf69d9b14953c78471722a, disabling compactions & flushes 2023-11-27 14:55:39,430 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:39,430 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:39,430 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. after waiting 0 ms 2023-11-27 14:55:39,430 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:39,430 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:39,430 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for f29516ec1eaf69d9b14953c78471722a: 2023-11-27 14:55:39,433 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:55:39,434 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701096939434"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096939434"}]},"ts":"1701096939434"} 2023-11-27 14:55:39,435 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-27 14:55:39,436 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:55:39,436 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096939436"}]},"ts":"1701096939436"} 2023-11-27 14:55:39,437 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-11-27 14:55:39,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f29516ec1eaf69d9b14953c78471722a, ASSIGN}] 2023-11-27 14:55:39,442 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f29516ec1eaf69d9b14953c78471722a, ASSIGN 2023-11-27 14:55:39,443 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=f29516ec1eaf69d9b14953c78471722a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:55:39,594 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=f29516ec1eaf69d9b14953c78471722a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:39,594 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701096939594"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096939594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096939594"}]},"ts":"1701096939594"} 2023-11-27 14:55:39,595 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure f29516ec1eaf69d9b14953c78471722a, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:39,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-27 14:55:39,747 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:39,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:39,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f29516ec1eaf69d9b14953c78471722a, NAME => 'testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:55:39,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-11-27 14:55:39,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-11-27 14:55:39,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:39,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:39,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:39,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:39,753 INFO [StoreOpener-f29516ec1eaf69d9b14953c78471722a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:39,754 DEBUG [StoreOpener-f29516ec1eaf69d9b14953c78471722a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1 2023-11-27 14:55:39,755 DEBUG [StoreOpener-f29516ec1eaf69d9b14953c78471722a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1 2023-11-27 14:55:39,755 INFO [StoreOpener-f29516ec1eaf69d9b14953c78471722a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f29516ec1eaf69d9b14953c78471722a columnFamilyName f1 2023-11-27 14:55:39,756 INFO [StoreOpener-f29516ec1eaf69d9b14953c78471722a-1] regionserver.HStore(310): Store=f29516ec1eaf69d9b14953c78471722a/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:39,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:39,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:39,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:39,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:55:39,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f29516ec1eaf69d9b14953c78471722a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62956735, jitterRate=-0.061871543526649475}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:55:39,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f29516ec1eaf69d9b14953c78471722a: 2023-11-27 14:55:39,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a., pid=73, masterSystemTime=1701096939747 2023-11-27 14:55:39,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:39,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:39,764 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=f29516ec1eaf69d9b14953c78471722a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:39,764 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701096939764"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096939764"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096939764"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096939764"}]},"ts":"1701096939764"} 2023-11-27 14:55:39,766 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=73, resume processing ppid=72 2023-11-27 14:55:39,766 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure f29516ec1eaf69d9b14953c78471722a, server=jenkins-hbase4.apache.org,42669,1701096850360 in 170 msec 2023-11-27 14:55:39,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=72, resume processing ppid=71 2023-11-27 14:55:39,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=f29516ec1eaf69d9b14953c78471722a, ASSIGN in 325 msec 2023-11-27 14:55:39,768 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:55:39,768 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096939768"}]},"ts":"1701096939768"} 2023-11-27 14:55:39,769 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-11-27 14:55:39,771 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:55:39,772 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1600 sec 2023-11-27 14:55:40,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-27 14:55:40,719 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-11-27 14:55:40,719 DEBUG [Listener at localhost/37439] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-11-27 14:55:40,721 INFO [Listener at localhost/37439] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-11-27 14:55:40,725 INFO [Listener at localhost/37439] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-11-27 14:55:40,725 INFO [Listener at localhost/37439] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-11-27 14:55:40,725 INFO [Listener at localhost/37439] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-11-27 14:55:40,729 INFO [Listener at localhost/37439] regionserver.HRegion(2745): Flushing f29516ec1eaf69d9b14953c78471722a 1/1 column families, dataSize=37 B heapSize=368 B 2023-11-27 14:55:41,144 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127e1f62a4fb4fe4684b8960d93fa3b5b38 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231127e1f62a4fb4fe4684b8960d93fa3b5b38 2023-11-27 14:55:41,550 INFO [Listener at localhost/37439] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/.tmp/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34 2023-11-27 14:55:41,556 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/.tmp/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34 2023-11-27 14:55:41,560 INFO [Listener at localhost/37439] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34, entries=1, sequenceid=5, filesize=4.9 K 2023-11-27 14:55:41,561 INFO [Listener at localhost/37439] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for f29516ec1eaf69d9b14953c78471722a in 832ms, sequenceid=5, compaction requested=false 2023-11-27 14:55:41,561 DEBUG [Listener at localhost/37439] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-11-27 14:55:41,562 DEBUG [Listener at localhost/37439] regionserver.HRegion(2446): Flush status journal for f29516ec1eaf69d9b14953c78471722a: 2023-11-27 14:55:41,564 INFO [Listener at localhost/37439] regionserver.HRegion(2745): Flushing f29516ec1eaf69d9b14953c78471722a 1/1 column families, dataSize=37 B heapSize=368 B 2023-11-27 14:55:41,975 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311274248b63c7d43483582827f4eb53c5ea2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202311274248b63c7d43483582827f4eb53c5ea2 2023-11-27 14:55:42,380 INFO [Listener at localhost/37439] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/.tmp/f1/21daa7d95c8b49398438e63d571ebb8e 2023-11-27 14:55:42,386 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/.tmp/f1/21daa7d95c8b49398438e63d571ebb8e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e 2023-11-27 14:55:42,390 INFO [Listener at localhost/37439] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e, entries=1, sequenceid=9, filesize=4.9 K 2023-11-27 14:55:42,391 INFO [Listener at localhost/37439] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for f29516ec1eaf69d9b14953c78471722a in 827ms, sequenceid=9, compaction requested=false 2023-11-27 14:55:42,391 DEBUG [Listener at localhost/37439] regionserver.HRegion(2446): Flush status journal for f29516ec1eaf69d9b14953c78471722a: 2023-11-27 14:55:42,393 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-11-27 14:55:42,395 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-27 14:55:42,396 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-27 14:55:42,396 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:42,396 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097542396 2023-11-27 14:55:42,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-27 14:55:42,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-11-27 14:55:42,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-11-27 14:55:42,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x05f45368 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:55:42,549 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:55:42,549 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-11-27 14:55:42,549 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3345192b to 127.0.0.1:61098 2023-11-27 14:55:42,549 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:55:42,549 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7cf997cc to 127.0.0.1:61098 2023-11-27 14:55:42,549 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:55:42,549 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:55:42,549 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x777b5fe7 to 127.0.0.1:61098 2023-11-27 14:55:42,549 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:55:42,550 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,550 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,556 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@31789ae8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:55:42,556 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231127 for table='testGetAfterCompaction' and column='f1' 2023-11-27 14:55:42,600 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,600 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,651 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,651 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,673 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:55:42,702 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,702 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,753 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,754 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,805 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,805 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,856 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,856 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,907 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,907 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,959 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:42,959 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,010 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,010 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,061 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,061 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,112 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,112 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,163 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,163 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,214 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,214 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,265 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,265 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,316 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,316 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,367 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,367 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,381 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112755f9985145bb475184f4e09e66a596de to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023112755f9985145bb475184f4e09e66a596de 2023-11-27 14:55:43,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='testGetAfterCompaction' column='f1' 2023-11-27 14:55:43,395 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:55:43,396 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57744, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:55:43,403 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:55:43,404 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37762, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:55:43,413 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231127/f1/57887fa4adb040acbdf9a7e452c0d0f6 first=Optional[r1] last=Optional[r2] 2023-11-27 14:55:43,413 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{f1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231127/f1/57887fa4adb040acbdf9a7e452c0d0f6}] 2023-11-27 14:55:43,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231127/f1/57887fa4adb040acbdf9a7e452c0d0f6 for inclusion in f29516ec1eaf69d9b14953c78471722a/f1 2023-11-27 14:55:43,418 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,418 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-11-27 14:55:43,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:55:43,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for f29516ec1eaf69d9b14953c78471722a: 2023-11-27 14:55:43,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231127/f1/57887fa4adb040acbdf9a7e452c0d0f6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testGetAfterCompaction__tmhng0vr141hel04t8j80lk81rshmsc8k3h8rkn8t9gum8ub3n94i8hi54ircgr/f1/57887fa4adb040acbdf9a7e452c0d0f6 2023-11-27 14:55:43,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testGetAfterCompaction__tmhng0vr141hel04t8j80lk81rshmsc8k3h8rkn8t9gum8ub3n94i8hi54ircgr/f1/57887fa4adb040acbdf9a7e452c0d0f6 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_ 2023-11-27 14:55:43,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testGetAfterCompaction__tmhng0vr141hel04t8j80lk81rshmsc8k3h8rkn8t9gum8ub3n94i8hi54ircgr/f1/57887fa4adb040acbdf9a7e452c0d0f6 into f29516ec1eaf69d9b14953c78471722a/f1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_ - updating store file list. 2023-11-27 14:55:43,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_ into f29516ec1eaf69d9b14953c78471722a/f1 2023-11-27 14:55:43,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testGetAfterCompaction__tmhng0vr141hel04t8j80lk81rshmsc8k3h8rkn8t9gum8ub3n94i8hi54ircgr/f1/57887fa4adb040acbdf9a7e452c0d0f6 into f29516ec1eaf69d9b14953c78471722a/f1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_) 2023-11-27 14:55:43,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testGetAfterCompaction__tmhng0vr141hel04t8j80lk81rshmsc8k3h8rkn8t9gum8ub3n94i8hi54ircgr/f1/57887fa4adb040acbdf9a7e452c0d0f6 2023-11-27 14:55:43,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): f29516ec1eaf69d9b14953c78471722a/f1 is initiating minor compaction 2023-11-27 14:55:43,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2fd9d0b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:43,427 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f29516ec1eaf69d9b14953c78471722a/f1 in testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:43,428 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/.tmp, totalSize=10.0 K 2023-11-27 14:55:43,428 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 21daa7d95c8b49398438e63d571ebb8e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-27 14:55:43,428 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 617970ec5ece4a10a4464974d9d71d0b_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-27 14:55:43,430 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='testGetAfterCompaction' column='f1' 2023-11-27 14:55:43,431 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:43,433 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202311274248b63c7d43483582827f4eb53c5ea2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202311274248b63c7d43483582827f4eb53c5ea2 2023-11-27 14:55:43,435 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): f29516ec1eaf69d9b14953c78471722a#f1#compaction#75 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:55:43,437 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231127e1f62a4fb4fe4684b8960d93fa3b5b38 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231127e1f62a4fb4fe4684b8960d93fa3b5b38 2023-11-27 14:55:43,437 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-11-27 14:55:43,437 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231127' 2023-11-27 14:55:43,437 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231127 for table='testGetAfterCompaction' and column='f1' 2023-11-27 14:55:43,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-11-27 14:55:43,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:55:43,441 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-27 14:55:43,443 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 1.0470 sec 2023-11-27 14:55:43,469 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,469 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,469 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,472 DEBUG [Listener at localhost/37439] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-27 14:55:43,476 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57756, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-27 14:55:43,476 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42669] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:43,477 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-11-27 14:55:43,477 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-27 14:55:43,477 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42669] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-27 14:55:43,477 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42669] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. because compaction request was cancelled 2023-11-27 14:55:43,528 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,528 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,528 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,579 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,579 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,579 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,630 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,630 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,630 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,681 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,681 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,681 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,732 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,732 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,732 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,783 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,783 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,783 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,834 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,834 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,834 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,852 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/.tmp/f1/97b167e57e554f3ca4835054094971ef as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/97b167e57e554f3ca4835054094971ef 2023-11-27 14:55:43,857 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in f29516ec1eaf69d9b14953c78471722a/f1 of f29516ec1eaf69d9b14953c78471722a into 97b167e57e554f3ca4835054094971ef(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:55:43,857 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f29516ec1eaf69d9b14953c78471722a: 2023-11-27 14:55:43,857 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1701096943427; duration=0sec 2023-11-27 14:55:43,857 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:55:43,885 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,885 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,885 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/97b167e57e554f3ca4835054094971ef; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1701096943847; access_time=1701096943432; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,885 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,936 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096942380; access_time=1701096941565; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,936 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1701096943377; access_time=1701096942568; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-27 14:55:43,936 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/97b167e57e554f3ca4835054094971ef; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1701096943847; access_time=1701096943432; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,936 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,949 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_] to archive 2023-11-27 14:55:43,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:43,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/21daa7d95c8b49398438e63d571ebb8e 2023-11-27 14:55:43,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/617970ec5ece4a10a4464974d9d71d0b_SeqId_12_ 2023-11-27 14:55:43,987 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/97b167e57e554f3ca4835054094971ef; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1701096943847; access_time=1701096943432; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:43,987 INFO [Listener at localhost/37439] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1701096941550; access_time=1701096940732; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-27 14:55:44,103 INFO [Listener at localhost/37439] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-11-27 14:55:44,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-11-27 14:55:44,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-11-27 14:55:44,115 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096944115"}]},"ts":"1701096944115"} 2023-11-27 14:55:44,117 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-11-27 14:55:44,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-27 14:55:44,118 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-11-27 14:55:44,119 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f29516ec1eaf69d9b14953c78471722a, UNASSIGN}] 2023-11-27 14:55:44,121 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f29516ec1eaf69d9b14953c78471722a, UNASSIGN 2023-11-27 14:55:44,121 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=f29516ec1eaf69d9b14953c78471722a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:44,121 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701096944121"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096944121"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096944121"}]},"ts":"1701096944121"} 2023-11-27 14:55:44,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure f29516ec1eaf69d9b14953c78471722a, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:44,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-27 14:55:44,274 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:44,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:44,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f29516ec1eaf69d9b14953c78471722a, disabling compactions & flushes 2023-11-27 14:55:44,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:44,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:44,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. after waiting 0 ms 2023-11-27 14:55:44,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:44,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-11-27 14:55:44,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-11-27 14:55:44,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a. 2023-11-27 14:55:44,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f29516ec1eaf69d9b14953c78471722a: 2023-11-27 14:55:44,281 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:44,282 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=f29516ec1eaf69d9b14953c78471722a, regionState=CLOSED 2023-11-27 14:55:44,282 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1701096944282"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096944282"}]},"ts":"1701096944282"} 2023-11-27 14:55:44,284 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=77, resume processing ppid=76 2023-11-27 14:55:44,284 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure f29516ec1eaf69d9b14953c78471722a, server=jenkins-hbase4.apache.org,42669,1701096850360 in 161 msec 2023-11-27 14:55:44,286 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=76, resume processing ppid=75 2023-11-27 14:55:44,286 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=f29516ec1eaf69d9b14953c78471722a, UNASSIGN in 165 msec 2023-11-27 14:55:44,286 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096944286"}]},"ts":"1701096944286"} 2023-11-27 14:55:44,287 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-11-27 14:55:44,289 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-11-27 14:55:44,290 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 180 msec 2023-11-27 14:55:44,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-27 14:55:44,419 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-11-27 14:55:44,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-11-27 14:55:44,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-11-27 14:55:44,431 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-27 14:55:44,432 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-27 14:55:44,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-11-27 14:55:44,435 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:44,437 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1, FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/recovered.edits] 2023-11-27 14:55:44,440 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/97b167e57e554f3ca4835054094971ef to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/97b167e57e554f3ca4835054094971ef 2023-11-27 14:55:44,441 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/f1/9f5a2e7c2b174f56a509b2ee2c7a9d34 2023-11-27 14:55:44,444 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/recovered.edits/15.seqid to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a/recovered.edits/15.seqid 2023-11-27 14:55:44,444 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testGetAfterCompaction/f29516ec1eaf69d9b14953c78471722a 2023-11-27 14:55:44,444 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-11-27 14:55:44,445 DEBUG [PEWorker-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-11-27 14:55:44,446 DEBUG [PEWorker-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-11-27 14:55:44,449 DEBUG [PEWorker-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023112755f9985145bb475184f4e09e66a596de to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023112755f9985145bb475184f4e09e66a596de 2023-11-27 14:55:44,449 DEBUG [PEWorker-1] backup.HFileArchiver(596): Deleted hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-11-27 14:55:44,451 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-27 14:55:44,473 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-11-27 14:55:44,475 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-11-27 14:55:44,476 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-27 14:55:44,476 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-11-27 14:55:44,476 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1701096944476"}]},"ts":"9223372036854775807"} 2023-11-27 14:55:44,478 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-11-27 14:55:44,478 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f29516ec1eaf69d9b14953c78471722a, NAME => 'testGetAfterCompaction,,1701096938611.f29516ec1eaf69d9b14953c78471722a.', STARTKEY => '', ENDKEY => ''}] 2023-11-27 14:55:44,478 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-11-27 14:55:44,478 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1701096944478"}]},"ts":"9223372036854775807"} 2023-11-27 14:55:44,479 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-11-27 14:55:44,481 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-27 14:55:44,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 56 msec 2023-11-27 14:55:44,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-11-27 14:55:44,535 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-11-27 14:55:44,547 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=239 (was 243), OpenFileDescriptor=630 (was 645), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 236), ProcessCount=168 (was 168), AvailableMemoryMB=8124 (was 8181) 2023-11-27 14:55:44,558 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=239, OpenFileDescriptor=630, MaxFileDescriptor=60000, SystemLoadAverage=225, ProcessCount=168, AvailableMemoryMB=8124 2023-11-27 14:55:44,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-27 14:55:44,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-11-27 14:55:44,562 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:55:44,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-11-27 14:55:44,562 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:55:44,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-27 14:55:44,565 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:44,565 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5 2023-11-27 14:55:44,565 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:44,565 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7 empty. 2023-11-27 14:55:44,565 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5 empty. 2023-11-27 14:55:44,565 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6 empty. 2023-11-27 14:55:44,566 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5 2023-11-27 14:55:44,566 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:44,566 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:44,566 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-11-27 14:55:44,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-27 14:55:44,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-27 14:55:44,975 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-11-27 14:55:44,976 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => d2db448818547f8516ae9f9e43b817b6, NAME => 'testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:55:44,977 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => f53154fc29f582902661898287c317b5, NAME => 'testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:55:44,977 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 68bb86a7752ed0247b952754e90231e7, NAME => 'testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:55:45,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-27 14:55:45,392 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing d2db448818547f8516ae9f9e43b817b6, disabling compactions & flushes 2023-11-27 14:55:45,393 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. after waiting 0 ms 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing f53154fc29f582902661898287c317b5, disabling compactions & flushes 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:45,393 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 68bb86a7752ed0247b952754e90231e7, disabling compactions & flushes 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:45,393 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. after waiting 0 ms 2023-11-27 14:55:45,393 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:55:45,393 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. after waiting 0 ms 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:45,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:45,393 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:45,394 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:55:45,396 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:55:45,397 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096945396"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096945396"}]},"ts":"1701096945396"} 2023-11-27 14:55:45,397 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701096945396"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096945396"}]},"ts":"1701096945396"} 2023-11-27 14:55:45,397 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096945396"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096945396"}]},"ts":"1701096945396"} 2023-11-27 14:55:45,399 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-27 14:55:45,399 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:55:45,399 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096945399"}]},"ts":"1701096945399"} 2023-11-27 14:55:45,401 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-11-27 14:55:45,406 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, ASSIGN}] 2023-11-27 14:55:45,407 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, ASSIGN 2023-11-27 14:55:45,407 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, ASSIGN 2023-11-27 14:55:45,407 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, ASSIGN 2023-11-27 14:55:45,407 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:55:45,408 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:55:45,408 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:55:45,560 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=68bb86a7752ed0247b952754e90231e7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:45,560 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=f53154fc29f582902661898287c317b5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:45,560 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=d2db448818547f8516ae9f9e43b817b6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:45,560 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096945560"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096945560"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096945560"}]},"ts":"1701096945560"} 2023-11-27 14:55:45,560 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701096945560"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096945560"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096945560"}]},"ts":"1701096945560"} 2023-11-27 14:55:45,560 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096945560"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096945560"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096945560"}]},"ts":"1701096945560"} 2023-11-27 14:55:45,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 68bb86a7752ed0247b952754e90231e7, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:45,562 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure f53154fc29f582902661898287c317b5, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:45,564 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure d2db448818547f8516ae9f9e43b817b6, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:55:45,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-27 14:55:45,714 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:45,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:45,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 68bb86a7752ed0247b952754e90231e7, NAME => 'testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.', STARTKEY => '', ENDKEY => '1'} 2023-11-27 14:55:45,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:45,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:45,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:45,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:45,719 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:45,721 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1 2023-11-27 14:55:45,721 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1 2023-11-27 14:55:45,721 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 68bb86a7752ed0247b952754e90231e7 columnFamilyName family1 2023-11-27 14:55:45,722 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(310): Store=68bb86a7752ed0247b952754e90231e7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:45,723 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:45,724 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2 2023-11-27 14:55:45,724 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2 2023-11-27 14:55:45,724 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 68bb86a7752ed0247b952754e90231e7 columnFamilyName family2 2023-11-27 14:55:45,725 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(310): Store=68bb86a7752ed0247b952754e90231e7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:45,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:45,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:45,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:55:45,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:55:45,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:55:45,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 68bb86a7752ed0247b952754e90231e7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62453286, jitterRate=-0.06937351822853088}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:55:45,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:55:45,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7., pid=83, masterSystemTime=1701096945714 2023-11-27 14:55:45,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:45,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:45,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:45,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d2db448818547f8516ae9f9e43b817b6, NAME => 'testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.', STARTKEY => '2', ENDKEY => ''} 2023-11-27 14:55:45,734 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=68bb86a7752ed0247b952754e90231e7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:45,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:45,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:45,734 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096945734"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096945734"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096945734"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096945734"}]},"ts":"1701096945734"} 2023-11-27 14:55:45,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:45,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:45,735 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:45,737 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1 2023-11-27 14:55:45,737 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1 2023-11-27 14:55:45,737 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=83, resume processing ppid=80 2023-11-27 14:55:45,737 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2db448818547f8516ae9f9e43b817b6 columnFamilyName family1 2023-11-27 14:55:45,737 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 68bb86a7752ed0247b952754e90231e7, server=jenkins-hbase4.apache.org,42669,1701096850360 in 174 msec 2023-11-27 14:55:45,738 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, ASSIGN in 333 msec 2023-11-27 14:55:45,738 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(310): Store=d2db448818547f8516ae9f9e43b817b6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:45,739 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:45,740 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2 2023-11-27 14:55:45,740 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2 2023-11-27 14:55:45,740 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2db448818547f8516ae9f9e43b817b6 columnFamilyName family2 2023-11-27 14:55:45,741 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(310): Store=d2db448818547f8516ae9f9e43b817b6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:45,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:45,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:45,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:55:45,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:55:45,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:55:45,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d2db448818547f8516ae9f9e43b817b6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65819937, jitterRate=-0.019206508994102478}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:55:45,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:55:45,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6., pid=85, masterSystemTime=1701096945714 2023-11-27 14:55:45,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:45,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:45,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:45,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f53154fc29f582902661898287c317b5, NAME => 'testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.', STARTKEY => '1', ENDKEY => '2'} 2023-11-27 14:55:45,750 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=d2db448818547f8516ae9f9e43b817b6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:45,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink f53154fc29f582902661898287c317b5 2023-11-27 14:55:45,750 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096945750"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096945750"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096945750"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096945750"}]},"ts":"1701096945750"} 2023-11-27 14:55:45,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:55:45,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f53154fc29f582902661898287c317b5 2023-11-27 14:55:45,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f53154fc29f582902661898287c317b5 2023-11-27 14:55:45,752 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f53154fc29f582902661898287c317b5 2023-11-27 14:55:45,753 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1 2023-11-27 14:55:45,753 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1 2023-11-27 14:55:45,753 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=85, resume processing ppid=82 2023-11-27 14:55:45,753 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure d2db448818547f8516ae9f9e43b817b6, server=jenkins-hbase4.apache.org,42669,1701096850360 in 188 msec 2023-11-27 14:55:45,753 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f53154fc29f582902661898287c317b5 columnFamilyName family1 2023-11-27 14:55:45,754 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(310): Store=f53154fc29f582902661898287c317b5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:45,755 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, ASSIGN in 348 msec 2023-11-27 14:55:45,755 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f53154fc29f582902661898287c317b5 2023-11-27 14:55:45,756 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2 2023-11-27 14:55:45,756 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2 2023-11-27 14:55:45,756 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f53154fc29f582902661898287c317b5 columnFamilyName family2 2023-11-27 14:55:45,757 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(310): Store=f53154fc29f582902661898287c317b5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:55:45,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5 2023-11-27 14:55:45,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5 2023-11-27 14:55:45,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:55:45,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f53154fc29f582902661898287c317b5 2023-11-27 14:55:45,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:55:45,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f53154fc29f582902661898287c317b5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64849198, jitterRate=-0.03367164731025696}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:55:45,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:45,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5., pid=84, masterSystemTime=1701096945714 2023-11-27 14:55:45,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:45,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:45,765 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=f53154fc29f582902661898287c317b5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:45,765 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701096945765"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096945765"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096945765"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096945765"}]},"ts":"1701096945765"} 2023-11-27 14:55:45,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=84, resume processing ppid=81 2023-11-27 14:55:45,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure f53154fc29f582902661898287c317b5, server=jenkins-hbase4.apache.org,42669,1701096850360 in 204 msec 2023-11-27 14:55:45,769 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=81, resume processing ppid=79 2023-11-27 14:55:45,769 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, ASSIGN in 364 msec 2023-11-27 14:55:45,769 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:55:45,770 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096945770"}]},"ts":"1701096945770"} 2023-11-27 14:55:45,771 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-11-27 14:55:45,773 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:55:45,774 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2130 sec 2023-11-27 14:55:46,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-27 14:55:46,668 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-11-27 14:55:46,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:46,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:46,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:46,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:46,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-27 14:55:46,682 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-27 14:55:46,682 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:46,682 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-27 14:55:46,682 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-27 14:55:46,682 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:46,682 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:46,683 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:46,683 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:46,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:46,684 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:46,684 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:46,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:46,684 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:46,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:46,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:46,685 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-27 14:55:46,685 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-27 14:55:46,685 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-27 14:55:46,685 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:46,685 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-27 14:55:46,685 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:46,685 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:55:46,685 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:46,685 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:46,685 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:46,685 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. started... 2023-11-27 14:55:46,685 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. started... 2023-11-27 14:55:46,685 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. started... 2023-11-27 14:55:46,686 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 68bb86a7752ed0247b952754e90231e7 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-27 14:55:46,687 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d2db448818547f8516ae9f9e43b817b6 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-27 14:55:46,687 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f53154fc29f582902661898287c317b5 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-27 14:55:47,107 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311273f6fd9351e9842df8dddeb3f1bd00e91 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311273f6fd9351e9842df8dddeb3f1bd00e91 2023-11-27 14:55:47,108 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127f4b90f9713bc44fc9fb9c60ce591c905 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127f4b90f9713bc44fc9fb9c60ce591c905 2023-11-27 14:55:47,111 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311278bd748d50ec449b591ab4997d2b6bb2a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311278bd748d50ec449b591ab4997d2b6bb2a 2023-11-27 14:55:47,126 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/9c5c6b42e7aa4057b2d2525439abb851 2023-11-27 14:55:47,140 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311275dfa46da3cbe4203997d262ff2da49ac to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311275dfa46da3cbe4203997d262ff2da49ac 2023-11-27 14:55:47,521 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/73d0bbf7e9cb43e3a1561e0e7e56f260 2023-11-27 14:55:47,527 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/11dbd4e159484772a2f6b1ba3218b84f 2023-11-27 14:55:47,535 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112725e6f6dea04343688bbff668571aadd3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112725e6f6dea04343688bbff668571aadd3 2023-11-27 14:55:47,545 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/aacaf93d059f41879ce21e42a7deb376 2023-11-27 14:55:47,548 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/9c5c6b42e7aa4057b2d2525439abb851 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/9c5c6b42e7aa4057b2d2525439abb851 2023-11-27 14:55:47,551 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/9c5c6b42e7aa4057b2d2525439abb851, entries=4, sequenceid=5, filesize=5.4 K 2023-11-27 14:55:47,552 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/aacaf93d059f41879ce21e42a7deb376 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/aacaf93d059f41879ce21e42a7deb376 2023-11-27 14:55:47,555 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/aacaf93d059f41879ce21e42a7deb376, entries=2, sequenceid=5, filesize=5.1 K 2023-11-27 14:55:47,556 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d2db448818547f8516ae9f9e43b817b6 in 868ms, sequenceid=5, compaction requested=false 2023-11-27 14:55:47,556 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-11-27 14:55:47,556 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:55:47,556 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:47,556 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:55:47,940 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/06188b30d7f149fc989a47c75248d5eb 2023-11-27 14:55:47,941 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127f1126d91cd6a4b99a8e2daddef1349a0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231127f1126d91cd6a4b99a8e2daddef1349a0 2023-11-27 14:55:47,945 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/73d0bbf7e9cb43e3a1561e0e7e56f260 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/73d0bbf7e9cb43e3a1561e0e7e56f260 2023-11-27 14:55:47,948 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/73d0bbf7e9cb43e3a1561e0e7e56f260, entries=4, sequenceid=5, filesize=5.4 K 2023-11-27 14:55:47,949 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/06188b30d7f149fc989a47c75248d5eb as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/06188b30d7f149fc989a47c75248d5eb 2023-11-27 14:55:47,953 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/06188b30d7f149fc989a47c75248d5eb, entries=2, sequenceid=5, filesize=5.1 K 2023-11-27 14:55:47,954 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f53154fc29f582902661898287c317b5 in 1267ms, sequenceid=5, compaction requested=false 2023-11-27 14:55:47,954 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:47,954 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:47,954 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:55:48,346 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/ac972ceb881e40c88492f6d98ccfde53 2023-11-27 14:55:48,351 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/11dbd4e159484772a2f6b1ba3218b84f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/11dbd4e159484772a2f6b1ba3218b84f 2023-11-27 14:55:48,354 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/11dbd4e159484772a2f6b1ba3218b84f, entries=4, sequenceid=5, filesize=5.4 K 2023-11-27 14:55:48,355 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/ac972ceb881e40c88492f6d98ccfde53 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/ac972ceb881e40c88492f6d98ccfde53 2023-11-27 14:55:48,358 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/ac972ceb881e40c88492f6d98ccfde53, entries=2, sequenceid=5, filesize=5.1 K 2023-11-27 14:55:48,359 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 68bb86a7752ed0247b952754e90231e7 in 1673ms, sequenceid=5, compaction requested=false 2023-11-27 14:55:48,359 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:55:48,359 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:48,359 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:55:48,359 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:55:48,360 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,360 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-27 14:55:48,360 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-27 14:55:48,362 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:48,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:48,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:48,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-27 14:55:48,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@730f51f4[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:48,364 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-27 14:55:48,364 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:48,365 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:48,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-27 14:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-27 14:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:48,365 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,365 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:48,366 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-27 14:55:48,367 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:48,366 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,367 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-27 14:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:48,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:48,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:48,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:48,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-27 14:55:48,369 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-27 14:55:48,369 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:48,369 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:48,369 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-27 14:55:48,369 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:48,370 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:48,370 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:48,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:48,371 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:48,371 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,372 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:48,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:48,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:48,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:48,375 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:48,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:48,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-27 14:55:48,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:48,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:48,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:48,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:48,376 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:48,376 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:48,376 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:48,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:48,476 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:48,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:48,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:48,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:48,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:48,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:48,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:48,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-27 14:55:48,483 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-27 14:55:48,483 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:48,483 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-27 14:55:48,483 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-27 14:55:48,483 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:48,483 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:48,485 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,485 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:48,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:48,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:48,485 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:48,485 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:48,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:48,485 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:48,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:48,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:48,486 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-27 14:55:48,486 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-27 14:55:48,486 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-27 14:55:48,486 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-27 14:55:48,486 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:48,486 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-27 14:55:48,487 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:48,487 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. started... 2023-11-27 14:55:48,487 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:48,487 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f53154fc29f582902661898287c317b5 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-27 14:55:48,487 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:48,487 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. started... 2023-11-27 14:55:48,487 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. started... 2023-11-27 14:55:48,487 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:55:48,487 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:48,487 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 68bb86a7752ed0247b952754e90231e7 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-27 14:55:48,487 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d2db448818547f8516ae9f9e43b817b6 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-27 14:55:48,502 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127ca141289f66b4b37a06eaec5c63297ae to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127ca141289f66b4b37a06eaec5c63297ae 2023-11-27 14:55:48,903 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311276c172b1bef3842fea773178819944ccb to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311276c172b1bef3842fea773178819944ccb 2023-11-27 14:55:48,903 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311274a4801ad97a94826a437fba4aaced078 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311274a4801ad97a94826a437fba4aaced078 2023-11-27 14:55:48,906 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/f3476c76367a4094959a000e438862a7 2023-11-27 14:55:48,908 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/54bff7e3315248878a316aabec8f5fb8 2023-11-27 14:55:48,926 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112703c1690fa86b4b6c800c5d46c68e7a7c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112703c1690fa86b4b6c800c5d46c68e7a7c 2023-11-27 14:55:49,308 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/44d0a27b9dfe46dcb2a46922bc5cb794 2023-11-27 14:55:49,330 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/62cd5b0e36874070bd39a34ab752b1a6 2023-11-27 14:55:49,330 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127728aca12e81d412bb59d2d32c500c398 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127728aca12e81d412bb59d2d32c500c398 2023-11-27 14:55:49,334 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/f3476c76367a4094959a000e438862a7 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f3476c76367a4094959a000e438862a7 2023-11-27 14:55:49,335 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/55af4ee02b754f3e80ab86a4399ba221 2023-11-27 14:55:49,338 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f3476c76367a4094959a000e438862a7, entries=4, sequenceid=9, filesize=5.4 K 2023-11-27 14:55:49,339 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/62cd5b0e36874070bd39a34ab752b1a6 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/62cd5b0e36874070bd39a34ab752b1a6 2023-11-27 14:55:49,339 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/54bff7e3315248878a316aabec8f5fb8 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/54bff7e3315248878a316aabec8f5fb8 2023-11-27 14:55:49,343 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/62cd5b0e36874070bd39a34ab752b1a6, entries=2, sequenceid=9, filesize=5.1 K 2023-11-27 14:55:49,343 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/54bff7e3315248878a316aabec8f5fb8, entries=4, sequenceid=9, filesize=5.4 K 2023-11-27 14:55:49,343 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 68bb86a7752ed0247b952754e90231e7 in 856ms, sequenceid=9, compaction requested=false 2023-11-27 14:55:49,343 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:55:49,343 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:55:49,343 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/55af4ee02b754f3e80ab86a4399ba221 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/55af4ee02b754f3e80ab86a4399ba221 2023-11-27 14:55:49,347 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/55af4ee02b754f3e80ab86a4399ba221, entries=2, sequenceid=9, filesize=5.1 K 2023-11-27 14:55:49,347 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d2db448818547f8516ae9f9e43b817b6 in 860ms, sequenceid=9, compaction requested=false 2023-11-27 14:55:49,348 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:55:49,348 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:49,348 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:55:49,723 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127f50035775c9a41d3ae3c6c36f321bc32 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231127f50035775c9a41d3ae3c6c36f321bc32 2023-11-27 14:55:49,781 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:55:50,128 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/f646c62f8dfd4699bf74c0db3d8ca297 2023-11-27 14:55:50,133 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/44d0a27b9dfe46dcb2a46922bc5cb794 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/44d0a27b9dfe46dcb2a46922bc5cb794 2023-11-27 14:55:50,136 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/44d0a27b9dfe46dcb2a46922bc5cb794, entries=4, sequenceid=9, filesize=5.4 K 2023-11-27 14:55:50,137 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/f646c62f8dfd4699bf74c0db3d8ca297 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/f646c62f8dfd4699bf74c0db3d8ca297 2023-11-27 14:55:50,140 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/f646c62f8dfd4699bf74c0db3d8ca297, entries=2, sequenceid=9, filesize=5.1 K 2023-11-27 14:55:50,141 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f53154fc29f582902661898287c317b5 in 1654ms, sequenceid=9, compaction requested=false 2023-11-27 14:55:50,141 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:50,141 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:55:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:55:50,141 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,142 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-27 14:55:50,142 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-27 14:55:50,146 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:50,146 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:50,146 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:50,146 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:50,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:50,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-27 14:55:50,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@599bf270[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:50,148 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-27 14:55:50,148 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:50,149 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:50,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:50,149 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-27 14:55:50,149 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-27 14:55:50,149 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:50,149 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,149 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:50,150 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:50,150 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-27 14:55:50,151 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:50,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:50,151 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-27 14:55:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:50,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:50,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-27 14:55:50,153 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-27 14:55:50,153 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:50,153 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:50,153 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-27 14:55:50,153 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:50,156 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:50,156 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:50,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:50,156 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:50,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:50,157 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:50,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:50,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:50,158 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:50,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,160 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:50,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:50,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-27 14:55:50,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:50,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-27 14:55:50,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:50,161 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:50,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:50,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:50,161 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:50,261 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:50,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:50,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:50,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:50,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:50,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:50,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:50,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-27 14:55:50,268 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-27 14:55:50,268 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:50,268 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-27 14:55:50,268 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-27 14:55:50,269 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:50,269 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:50,270 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,270 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:50,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:50,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:50,270 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:50,271 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:50,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:50,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:50,272 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-27 14:55:50,272 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-27 14:55:50,272 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-27 14:55:50,272 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-27 14:55:50,272 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:50,272 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-27 14:55:50,272 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:50,272 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. started... 2023-11-27 14:55:50,272 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:50,272 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:55:50,272 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:50,272 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f53154fc29f582902661898287c317b5 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-27 14:55:50,272 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. started... 2023-11-27 14:55:50,272 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:50,273 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d2db448818547f8516ae9f9e43b817b6 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-27 14:55:50,273 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. started... 2023-11-27 14:55:50,273 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 68bb86a7752ed0247b952754e90231e7 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-27 14:55:50,690 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311279f6d46bf55ff44f8a38c99407a29d75f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311279f6d46bf55ff44f8a38c99407a29d75f 2023-11-27 14:55:50,697 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112743713d96bf2d4fa7878b42c12e1a7fe5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112743713d96bf2d4fa7878b42c12e1a7fe5 2023-11-27 14:55:50,698 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127ae5335fb8ba24277a60870ae7f9a4267 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ae5335fb8ba24277a60870ae7f9a4267 2023-11-27 14:55:51,098 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/83ab5ebd1f744748a511df6081b4ca4e 2023-11-27 14:55:51,111 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/8bb35dffb04542408445e0a661ed6843 2023-11-27 14:55:51,112 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/6cae8ebbf33443cea0a04ac326c2c78e 2023-11-27 14:55:51,513 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311274996160064294a1195a80e166a6d7d89 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311274996160064294a1195a80e166a6d7d89 2023-11-27 14:55:51,525 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311277de6939e72d543e5abb705afb4a0f052 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311277de6939e72d543e5abb705afb4a0f052 2023-11-27 14:55:51,525 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112736f5012b8d4b4c318a8dcd007b924aca to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112736f5012b8d4b4c318a8dcd007b924aca 2023-11-27 14:55:51,918 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/7cade81205724088b52161b761cad33c 2023-11-27 14:55:51,923 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/83ab5ebd1f744748a511df6081b4ca4e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/83ab5ebd1f744748a511df6081b4ca4e 2023-11-27 14:55:51,926 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/83ab5ebd1f744748a511df6081b4ca4e, entries=4, sequenceid=13, filesize=5.4 K 2023-11-27 14:55:51,927 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/7cade81205724088b52161b761cad33c as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/7cade81205724088b52161b761cad33c 2023-11-27 14:55:51,930 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/3f82744f5963467498262e333eab0bca 2023-11-27 14:55:51,931 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/be1d45f3a67c4f3eaeea2cd5482139f0 2023-11-27 14:55:51,931 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/7cade81205724088b52161b761cad33c, entries=2, sequenceid=13, filesize=5.1 K 2023-11-27 14:55:51,932 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d2db448818547f8516ae9f9e43b817b6 in 1659ms, sequenceid=13, compaction requested=true 2023-11-27 14:55:51,932 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:55:51,932 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:51,932 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:55:51,935 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/8bb35dffb04542408445e0a661ed6843 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/8bb35dffb04542408445e0a661ed6843 2023-11-27 14:55:51,935 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/6cae8ebbf33443cea0a04ac326c2c78e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/6cae8ebbf33443cea0a04ac326c2c78e 2023-11-27 14:55:51,938 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/8bb35dffb04542408445e0a661ed6843, entries=4, sequenceid=13, filesize=5.4 K 2023-11-27 14:55:51,939 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/6cae8ebbf33443cea0a04ac326c2c78e, entries=4, sequenceid=13, filesize=5.4 K 2023-11-27 14:55:51,939 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/be1d45f3a67c4f3eaeea2cd5482139f0 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/be1d45f3a67c4f3eaeea2cd5482139f0 2023-11-27 14:55:51,939 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/3f82744f5963467498262e333eab0bca as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/3f82744f5963467498262e333eab0bca 2023-11-27 14:55:51,943 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/be1d45f3a67c4f3eaeea2cd5482139f0, entries=2, sequenceid=13, filesize=5.1 K 2023-11-27 14:55:51,943 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/3f82744f5963467498262e333eab0bca, entries=2, sequenceid=13, filesize=5.1 K 2023-11-27 14:55:51,944 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f53154fc29f582902661898287c317b5 in 1672ms, sequenceid=13, compaction requested=true 2023-11-27 14:55:51,944 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 68bb86a7752ed0247b952754e90231e7 in 1671ms, sequenceid=13, compaction requested=true 2023-11-27 14:55:51,944 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:51,944 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:55:51,944 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:51,944 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:51,944 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:55:51,944 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:55:51,944 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:55:51,944 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,944 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-27 14:55:51,944 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-27 14:55:51,946 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:51,946 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:51,946 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:51,946 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:51,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:51,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:51,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:51,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:51,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-27 14:55:51,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@773b1db4[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:51,948 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-27 14:55:51,948 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:51,950 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:51,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:51,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:51,951 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-27 14:55:51,951 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-27 14:55:51,951 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:51,951 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,951 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:51,952 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,952 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-27 14:55:51,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,952 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:51,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:51,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:51,952 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-27 14:55:51,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:51,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:51,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:51,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-27 14:55:51,955 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-27 14:55:51,955 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:51,955 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:51,955 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-27 14:55:51,955 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:51,956 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:51,956 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:51,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:51,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:51,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:51,957 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:51,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:51,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:51,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:51,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:51,957 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:51,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:51,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:51,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:51,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:51,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:51,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:51,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:51,961 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:51,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-27 14:55:51,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:51,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:51,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:51,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:51,961 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:51,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:51,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:52,061 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:52,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:52,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:52,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:52,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:55:52,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:52,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:52,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-27 14:55:52,069 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-27 14:55:52,069 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:52,069 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-27 14:55:52,069 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-27 14:55:52,070 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:52,070 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:52,072 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:52,072 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:52,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:52,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:52,072 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:52,072 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:52,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:52,073 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:52,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:52,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:52,073 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-27 14:55:52,074 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-27 14:55:52,074 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-27 14:55:52,074 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-27 14:55:52,074 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:52,074 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-27 14:55:52,075 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:52,075 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. started... 2023-11-27 14:55:52,075 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f53154fc29f582902661898287c317b5 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-27 14:55:52,075 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:52,075 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. started... 2023-11-27 14:55:52,075 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d2db448818547f8516ae9f9e43b817b6 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-27 14:55:52,076 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:55:52,076 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:52,076 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:52,076 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. started... 2023-11-27 14:55:52,076 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 68bb86a7752ed0247b952754e90231e7 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-27 14:55:52,498 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127fedadc64d2504dccb03740338659f0c6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127fedadc64d2504dccb03740338659f0c6 2023-11-27 14:55:52,502 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127ecf30d17888a4c878b4e64030723965f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ecf30d17888a4c878b4e64030723965f 2023-11-27 14:55:52,505 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127deae02fa52974d90841c7ec64ee944df to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127deae02fa52974d90841c7ec64ee944df 2023-11-27 14:55:52,512 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/19921ab818304b7ca96849e3388bd514 2023-11-27 14:55:52,529 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311276b40aff22efd47b288a2aca766bd3f97 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311276b40aff22efd47b288a2aca766bd3f97 2023-11-27 14:55:52,908 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/e77c71cc85014290a851fe41a0dc07ba 2023-11-27 14:55:52,914 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e 2023-11-27 14:55:52,933 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/384e3e9c79f642ecaf650fef248f348e 2023-11-27 14:55:52,937 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/19921ab818304b7ca96849e3388bd514 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/19921ab818304b7ca96849e3388bd514 2023-11-27 14:55:52,941 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/19921ab818304b7ca96849e3388bd514, entries=4, sequenceid=17, filesize=5.4 K 2023-11-27 14:55:52,941 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/384e3e9c79f642ecaf650fef248f348e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/384e3e9c79f642ecaf650fef248f348e 2023-11-27 14:55:52,945 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/384e3e9c79f642ecaf650fef248f348e, entries=2, sequenceid=17, filesize=5.1 K 2023-11-27 14:55:52,946 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f53154fc29f582902661898287c317b5 in 871ms, sequenceid=17, compaction requested=true 2023-11-27 14:55:52,946 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:52,946 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:52,946 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:55:53,322 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127018167e6893d4b1d9cdc8a9dcec1aa7d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127018167e6893d4b1d9cdc8a9dcec1aa7d 2023-11-27 14:55:53,328 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311272d64f381bd574ed7a0b3846e5a11f70a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311272d64f381bd574ed7a0b3846e5a11f70a 2023-11-27 14:55:53,731 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/383ab85ed5fa46a19088c45d22cb1c6f 2023-11-27 14:55:53,733 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/7362e669758d4b249076960802af5b98 2023-11-27 14:55:53,736 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/e77c71cc85014290a851fe41a0dc07ba as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e77c71cc85014290a851fe41a0dc07ba 2023-11-27 14:55:53,737 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e 2023-11-27 14:55:53,740 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e77c71cc85014290a851fe41a0dc07ba, entries=4, sequenceid=17, filesize=5.4 K 2023-11-27 14:55:53,740 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/383ab85ed5fa46a19088c45d22cb1c6f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/383ab85ed5fa46a19088c45d22cb1c6f 2023-11-27 14:55:53,742 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e, entries=4, sequenceid=17, filesize=5.4 K 2023-11-27 14:55:53,743 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/7362e669758d4b249076960802af5b98 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/7362e669758d4b249076960802af5b98 2023-11-27 14:55:53,745 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/383ab85ed5fa46a19088c45d22cb1c6f, entries=2, sequenceid=17, filesize=5.1 K 2023-11-27 14:55:53,745 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d2db448818547f8516ae9f9e43b817b6 in 1670ms, sequenceid=17, compaction requested=true 2023-11-27 14:55:53,746 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:55:53,746 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:53,746 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:55:53,747 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/7362e669758d4b249076960802af5b98, entries=2, sequenceid=17, filesize=5.1 K 2023-11-27 14:55:53,748 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 68bb86a7752ed0247b952754e90231e7 in 1672ms, sequenceid=17, compaction requested=true 2023-11-27 14:55:53,748 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:55:53,748 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:53,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:55:53,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:55:53,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-27 14:55:53,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-27 14:55:53,750 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:53,750 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:53,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:53,750 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:53,750 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:53,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:53,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:53,751 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-27 14:55:53,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50e3e2c7[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:53,752 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-27 14:55:53,752 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:53,753 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:53,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:53,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-27 14:55:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-27 14:55:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:53,753 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,753 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:53,754 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,754 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-27 14:55:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:53,754 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:53,754 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-27 14:55:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:53,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:53,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:53,757 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-27 14:55:53,757 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:53,757 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:53,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-27 14:55:53,757 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-27 14:55:53,757 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:53,759 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:53,759 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:53,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:53,759 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:53,760 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:53,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:53,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:53,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,761 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:53,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:53,764 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:53,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-27 14:55:53,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:53,764 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:53,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:53,864 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:53,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:53,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-27 14:55:53,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1701096953876 (current time:1701096953876). 2023-11-27 14:55:53,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-11-27 14:55:53,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] zookeeper.ReadOnlyZKClient(139): Connect 0x799805e1 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:55:53,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15341a6d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:55:53,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:55:53,891 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49724, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:55:53,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x799805e1 to 127.0.0.1:61098 2023-11-27 14:55:53,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:55:53,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-11-27 14:55:53,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotManager(684): Table enabled, starting distributed snapshots for { ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-27 14:55:53,903 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot, current working dir path: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/snaptb-1701096953866 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotEnabledTable(SnapshotManager.java:559) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:687) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-11-27 14:55:53,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-27 14:55:53,911 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-27 14:55:53,912 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1701097553912 2023-11-27 14:55:53,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotManager(689): Started snapshot: { ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-27 14:55:53,912 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1701096953866 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-11-27 14:55:53,914 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-27 14:55:53,914 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1028): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-27 14:55:53,915 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 4 msec 2023-11-27 14:55:53,915 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-27 14:55:53,916 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1701097553916 2023-11-27 14:55:53,917 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:53,918 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-11-27 14:55:53,924 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1701096953866 2023-11-27 14:55:53,928 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1701096953866' 2023-11-27 14:55:53,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-11-27 14:55:53,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1701096953866' starting 'acquire' 2023-11-27 14:55:53,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1701096953866', kicking off acquire phase on members. 2023-11-27 14:55:53,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1701096953866 2023-11-27 14:55:53,928 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1701096953866 2023-11-27 14:55:53,929 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-11-27 14:55:53,929 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-11-27 14:55:53,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-27 14:55:53,930 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,930 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:53,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1701096953866 2023-11-27 14:55:53,930 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1701096953866 2023-11-27 14:55:53,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-11-27 14:55:53,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1701096953866 2023-11-27 14:55:53,931 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1701096953866 from table testCompactionWithHFileLink type FLUSH 2023-11-27 14:55:53,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1701096953866 2023-11-27 14:55:53,934 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1701096953866' with timeout 300000ms 2023-11-27 14:55:53,934 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-11-27 14:55:53,934 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1701096953866' starting 'acquire' stage 2023-11-27 14:55:53,934 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1701096953866' locally acquired 2023-11-27 14:55:53,934 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (snaptb-1701096953866) in zk 2023-11-27 14:55:53,935 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,935 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1701096953866 2023-11-27 14:55:53,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:53,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-27 14:55:53,936 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1701096953866 2023-11-27 14:55:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:53,936 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1701096953866' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701096953866 2023-11-27 14:55:53,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:53,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'snaptb-1701096953866' on coordinator 2023-11-27 14:55:53,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@596a99de[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:53,937 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1701096953866' starting 'in-barrier' execution. 2023-11-27 14:55:53,937 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1701096953866 2023-11-27 14:55:53,939 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701096953866 2023-11-27 14:55:53,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1701096953866 2023-11-27 14:55:53,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1701096953866 2023-11-27 14:55:53,939 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1701096953866' received 'reached' from coordinator. 2023-11-27 14:55:53,939 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:53,939 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:53,940 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:53,940 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:53,940 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-11-27 14:55:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-11-27 14:55:53,941 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. started... 2023-11-27 14:55:53,941 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. started... 2023-11-27 14:55:53,941 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. started... 2023-11-27 14:55:53,941 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:55:53,941 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:53,941 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:55:53,942 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.' region-info for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,942 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.' region-info for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,942 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.' region-info for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,946 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-27 14:55:53,946 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-27 14:55:53,946 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-27 14:55:53,950 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/9c5c6b42e7aa4057b2d2525439abb851, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/54bff7e3315248878a316aabec8f5fb8, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/83ab5ebd1f744748a511df6081b4ca4e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e77c71cc85014290a851fe41a0dc07ba] hfiles 2023-11-27 14:55:53,950 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/73d0bbf7e9cb43e3a1561e0e7e56f260, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/44d0a27b9dfe46dcb2a46922bc5cb794, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/8bb35dffb04542408445e0a661ed6843, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/19921ab818304b7ca96849e3388bd514] hfiles 2023-11-27 14:55:53,950 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/11dbd4e159484772a2f6b1ba3218b84f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f3476c76367a4094959a000e438862a7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/6cae8ebbf33443cea0a04ac326c2c78e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e] hfiles 2023-11-27 14:55:53,950 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/73d0bbf7e9cb43e3a1561e0e7e56f260 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,950 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/9c5c6b42e7aa4057b2d2525439abb851 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,950 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/11dbd4e159484772a2f6b1ba3218b84f for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,955 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/54bff7e3315248878a316aabec8f5fb8 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,955 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f3476c76367a4094959a000e438862a7 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,955 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/44d0a27b9dfe46dcb2a46922bc5cb794 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/6cae8ebbf33443cea0a04ac326c2c78e for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/83ab5ebd1f744748a511df6081b4ca4e for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/8bb35dffb04542408445e0a661ed6843 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e77c71cc85014290a851fe41a0dc07ba for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/ac972ceb881e40c88492f6d98ccfde53, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/62cd5b0e36874070bd39a34ab752b1a6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/3f82744f5963467498262e333eab0bca, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/7362e669758d4b249076960802af5b98] hfiles 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/aacaf93d059f41879ce21e42a7deb376, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/55af4ee02b754f3e80ab86a4399ba221, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/7cade81205724088b52161b761cad33c, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/383ab85ed5fa46a19088c45d22cb1c6f] hfiles 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/aacaf93d059f41879ce21e42a7deb376 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/55af4ee02b754f3e80ab86a4399ba221 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/7cade81205724088b52161b761cad33c for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/19921ab818304b7ca96849e3388bd514 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/383ab85ed5fa46a19088c45d22cb1c6f for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/ac972ceb881e40c88492f6d98ccfde53 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/06188b30d7f149fc989a47c75248d5eb, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/f646c62f8dfd4699bf74c0db3d8ca297, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/be1d45f3a67c4f3eaeea2cd5482139f0, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/384e3e9c79f642ecaf650fef248f348e] hfiles 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/62cd5b0e36874070bd39a34ab752b1a6 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/06188b30d7f149fc989a47c75248d5eb for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/3f82744f5963467498262e333eab0bca for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/f646c62f8dfd4699bf74c0db3d8ca297 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,956 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/7362e669758d4b249076960802af5b98 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,957 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/be1d45f3a67c4f3eaeea2cd5482139f0 for snapshot=snaptb-1701096953866 2023-11-27 14:55:53,957 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/384e3e9c79f642ecaf650fef248f348e for snapshot=snaptb-1701096953866 2023-11-27 14:55:54,018 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:55:54,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:55:54,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-27 14:55:54,022 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-11-27 14:55:54,222 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:55:54,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:55:54,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-27 14:55:54,223 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-11-27 14:55:54,366 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. completed. 2023-11-27 14:55:54,366 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. completed. 2023-11-27 14:55:54,366 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:54,366 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:54,366 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-11-27 14:55:54,366 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-11-27 14:55:54,370 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. completed. 2023-11-27 14:55:54,370 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:54,370 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-11-27 14:55:54,370 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-11-27 14:55:54,370 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,370 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1701096953866' locally completed 2023-11-27 14:55:54,370 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1701096953866' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:54,372 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,372 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1701096953866' has notified controller of completion 2023-11-27 14:55:54,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:54,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-27 14:55:54,372 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:54,372 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1701096953866' completed. 2023-11-27 14:55:54,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:54,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:54,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701096953866 2023-11-27 14:55:54,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:54,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701096953866 2023-11-27 14:55:54,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1701096953866' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:54,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'snaptb-1701096953866', counting down latch. Waiting for 0 more 2023-11-27 14:55:54,375 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1701096953866' execution completed 2023-11-27 14:55:54,375 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:54,375 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:54,375 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1701096953866 2023-11-27 14:55:54,375 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1701096953866including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-11-27 14:55:54,376 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1701096953866 2023-11-27 14:55:54,376 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1701096953866 2023-11-27 14:55:54,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1701096953866 2023-11-27 14:55:54,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1701096953866 2023-11-27 14:55:54,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:54,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-27 14:55:54,376 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-11-27 14:55:54,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1701096953866 2023-11-27 14:55:54,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-11-27 14:55:54,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:54,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-27 14:55:54,377 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1701096953866 2023-11-27 14:55:54,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701096953866 2023-11-27 14:55:54,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:54,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701096953866 2023-11-27 14:55:54,378 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:54,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1701096953866 2023-11-27 14:55:54,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,380 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,380 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1701096953866 2023-11-27 14:55:54,380 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1701096953866 2023-11-27 14:55:54,380 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701096953866/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:54,380 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701096953866 2023-11-27 14:55:54,380 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:54,380 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1701096953866 2023-11-27 14:55:54,380 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-11-27 14:55:54,381 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-11-27 14:55:54,381 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-11-27 14:55:54,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-11-27 14:55:54,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-27 14:55:54,380 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1701096953866 because node does not exist (not an error) 2023-11-27 14:55:54,380 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1701096953866 2023-11-27 14:55:54,381 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1701096953866 2023-11-27 14:55:54,381 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1701096953866 2023-11-27 14:55:54,381 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-11-27 14:55:54,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-11-27 14:55:54,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-27 14:55:54,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311276c172b1bef3842fea773178819944ccb, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ae5335fb8ba24277a60870ae7f9a4267, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ecf30d17888a4c878b4e64030723965f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127f4b90f9713bc44fc9fb9c60ce591c905, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311273f6fd9351e9842df8dddeb3f1bd00e91, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311274a4801ad97a94826a437fba4aaced078, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311279f6d46bf55ff44f8a38c99407a29d75f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127fedadc64d2504dccb03740338659f0c6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112743713d96bf2d4fa7878b42c12e1a7fe5, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311278bd748d50ec449b591ab4997d2b6bb2a, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127ca141289f66b4b37a06eaec5c63297ae, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127deae02fa52974d90841c7ec64ee944df] mob files 2023-11-27 14:55:54,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311276c172b1bef3842fea773178819944ccb 2023-11-27 14:55:54,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ae5335fb8ba24277a60870ae7f9a4267 2023-11-27 14:55:54,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ecf30d17888a4c878b4e64030723965f 2023-11-27 14:55:54,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127f4b90f9713bc44fc9fb9c60ce591c905 2023-11-27 14:55:54,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311273f6fd9351e9842df8dddeb3f1bd00e91 2023-11-27 14:55:54,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311274a4801ad97a94826a437fba4aaced078 2023-11-27 14:55:54,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311279f6d46bf55ff44f8a38c99407a29d75f 2023-11-27 14:55:54,383 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127fedadc64d2504dccb03740338659f0c6 2023-11-27 14:55:54,383 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112743713d96bf2d4fa7878b42c12e1a7fe5 2023-11-27 14:55:54,383 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311278bd748d50ec449b591ab4997d2b6bb2a 2023-11-27 14:55:54,383 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127ca141289f66b4b37a06eaec5c63297ae 2023-11-27 14:55:54,383 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127deae02fa52974d90841c7ec64ee944df 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112725e6f6dea04343688bbff668571aadd3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112736f5012b8d4b4c318a8dcd007b924aca, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311276b40aff22efd47b288a2aca766bd3f97, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231127f50035775c9a41d3ae3c6c36f321bc32, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127018167e6893d4b1d9cdc8a9dcec1aa7d, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311274996160064294a1195a80e166a6d7d89, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311275dfa46da3cbe4203997d262ff2da49ac, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127728aca12e81d412bb59d2d32c500c398, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112703c1690fa86b4b6c800c5d46c68e7a7c, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311272d64f381bd574ed7a0b3846e5a11f70a, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311277de6939e72d543e5abb705afb4a0f052, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231127f1126d91cd6a4b99a8e2daddef1349a0] mob files 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112725e6f6dea04343688bbff668571aadd3 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112736f5012b8d4b4c318a8dcd007b924aca 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311276b40aff22efd47b288a2aca766bd3f97 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231127f50035775c9a41d3ae3c6c36f321bc32 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127018167e6893d4b1d9cdc8a9dcec1aa7d 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311274996160064294a1195a80e166a6d7d89 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311275dfa46da3cbe4203997d262ff2da49ac 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127728aca12e81d412bb59d2d32c500c398 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112703c1690fa86b4b6c800c5d46c68e7a7c 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311272d64f381bd574ed7a0b3846e5a11f70a 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311277de6939e72d543e5abb705afb4a0f052 2023-11-27 14:55:54,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231127f1126d91cd6a4b99a8e2daddef1349a0 2023-11-27 14:55:54,524 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:55:54,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:55:54,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-27 14:55:54,525 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-11-27 14:55:54,789 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1701096953866 2023-11-27 14:55:54,791 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/snaptb-1701096953866 2023-11-27 14:55:55,025 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:55:55,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:55:55,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-27 14:55:55,026 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-11-27 14:55:55,247 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/snaptb-1701096953866 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/snaptb-1701096953866 2023-11-27 14:55:55,247 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1701096953866 of table testCompactionWithHFileLink completed 2023-11-27 14:55:55,248 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/snaptb-1701096953866 2023-11-27 14:55:55,248 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/snaptb-1701096953866 2023-11-27 14:55:55,248 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1701096953866 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1701096953912Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1701096954789 (+877 ms)Consolidate snapshot: snaptb-1701096953866 at 1701096954789Loading Region manifests for snaptb-1701096953866 at 1701096954789Writing data manifest for snaptb-1701096953866 at 1701096954801 (+12 ms)Verifying snapshot: snaptb-1701096953866 at 1701096955210 (+409 ms)Snapshot snaptb-1701096953866 of table testCompactionWithHFileLink completed at 1701096955247 (+37 ms) 2023-11-27 14:55:55,249 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-27 14:55:55,251 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.3350 sec 2023-11-27 14:55:56,026 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:55:56,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:55:56,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] snapshot.SnapshotManager(403): Snapshot '{ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-11-27 14:55:56,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:55:56,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-27 14:55:56,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-27 14:55:56,063 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-27 14:55:56,063 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:56,063 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-27 14:55:56,063 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-27 14:55:56,063 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:56,063 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:56,065 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:56,065 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:56,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:56,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:56,065 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:56,065 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:55:56,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:56,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:56,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:55:56,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:56,066 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-27 14:55:56,066 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-27 14:55:56,066 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-27 14:55:56,066 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-27 14:55:56,066 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:55:56,066 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-27 14:55:56,067 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:56,067 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. started... 2023-11-27 14:55:56,067 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:56,067 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. started... 2023-11-27 14:55:56,067 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f53154fc29f582902661898287c317b5 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-27 14:55:56,067 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d2db448818547f8516ae9f9e43b817b6 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-27 14:55:56,067 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:55:56,067 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:56,067 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:55:56,067 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. started... 2023-11-27 14:55:56,067 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 68bb86a7752ed0247b952754e90231e7 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-27 14:55:56,099 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/3e099553dd574bd7b9536c0f396d3790 2023-11-27 14:55:56,103 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3e099553dd574bd7b9536c0f396d3790 2023-11-27 14:55:56,514 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/50220b22c7d1438d89f660702790888c 2023-11-27 14:55:56,517 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 50220b22c7d1438d89f660702790888c 2023-11-27 14:55:56,518 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/3e099553dd574bd7b9536c0f396d3790 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/3e099553dd574bd7b9536c0f396d3790 2023-11-27 14:55:56,521 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3e099553dd574bd7b9536c0f396d3790 2023-11-27 14:55:56,521 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/3e099553dd574bd7b9536c0f396d3790, entries=3, sequenceid=24, filesize=5.0 K 2023-11-27 14:55:56,522 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/50220b22c7d1438d89f660702790888c as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/50220b22c7d1438d89f660702790888c 2023-11-27 14:55:56,525 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 50220b22c7d1438d89f660702790888c 2023-11-27 14:55:56,526 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/50220b22c7d1438d89f660702790888c, entries=1, sequenceid=24, filesize=5.0 K 2023-11-27 14:55:56,526 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f53154fc29f582902661898287c317b5 in 459ms, sequenceid=24, compaction requested=true 2023-11-27 14:55:56,526 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:56,526 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:56,526 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:55:56,890 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/788c8703e81c49ca807a4a429c2dd649 2023-11-27 14:55:56,894 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 788c8703e81c49ca807a4a429c2dd649 2023-11-27 14:55:56,895 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/08d68f94afe44aa48e17cb61b1ef3c14 2023-11-27 14:55:56,899 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08d68f94afe44aa48e17cb61b1ef3c14 2023-11-27 14:55:57,310 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/8a0a5b2399114ecd83e6fa3a6174c58c 2023-11-27 14:55:57,311 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/49498639cd29446a946d001146acac2f 2023-11-27 14:55:57,314 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8a0a5b2399114ecd83e6fa3a6174c58c 2023-11-27 14:55:57,315 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 49498639cd29446a946d001146acac2f 2023-11-27 14:55:57,315 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/788c8703e81c49ca807a4a429c2dd649 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/788c8703e81c49ca807a4a429c2dd649 2023-11-27 14:55:57,316 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/08d68f94afe44aa48e17cb61b1ef3c14 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/08d68f94afe44aa48e17cb61b1ef3c14 2023-11-27 14:55:57,319 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08d68f94afe44aa48e17cb61b1ef3c14 2023-11-27 14:55:57,319 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/08d68f94afe44aa48e17cb61b1ef3c14, entries=3, sequenceid=24, filesize=5.0 K 2023-11-27 14:55:57,320 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 788c8703e81c49ca807a4a429c2dd649 2023-11-27 14:55:57,320 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/788c8703e81c49ca807a4a429c2dd649, entries=3, sequenceid=24, filesize=5.0 K 2023-11-27 14:55:57,320 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/49498639cd29446a946d001146acac2f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/49498639cd29446a946d001146acac2f 2023-11-27 14:55:57,321 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/8a0a5b2399114ecd83e6fa3a6174c58c as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/8a0a5b2399114ecd83e6fa3a6174c58c 2023-11-27 14:55:57,324 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 49498639cd29446a946d001146acac2f 2023-11-27 14:55:57,324 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/49498639cd29446a946d001146acac2f, entries=1, sequenceid=24, filesize=5.0 K 2023-11-27 14:55:57,325 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8a0a5b2399114ecd83e6fa3a6174c58c 2023-11-27 14:55:57,325 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/8a0a5b2399114ecd83e6fa3a6174c58c, entries=1, sequenceid=24, filesize=5.0 K 2023-11-27 14:55:57,325 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 68bb86a7752ed0247b952754e90231e7 in 1258ms, sequenceid=24, compaction requested=true 2023-11-27 14:55:57,325 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:55:57,325 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:55:57,325 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:55:57,325 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d2db448818547f8516ae9f9e43b817b6 in 1258ms, sequenceid=24, compaction requested=true 2023-11-27 14:55:57,325 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:55:57,326 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:55:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:55:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-27 14:55:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-27 14:55:57,327 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:57,327 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:57,328 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:57,328 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:55:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:57,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:57,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-27 14:55:57,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f64e8f1[Count = 0] remaining members to acquire global barrier 2023-11-27 14:55:57,329 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-27 14:55:57,329 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:57,330 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:57,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:57,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:57,331 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-27 14:55:57,331 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-27 14:55:57,331 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:55:57,331 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,331 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:55:57,333 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,333 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-27 14:55:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:57,333 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:57,333 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-27 14:55:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:57,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:55:57,335 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-27 14:55:57,335 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-27 14:55:57,335 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:55:57,335 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:55:57,335 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-27 14:55:57,335 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:55:57,336 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:57,336 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:55:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:55:57,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:57,336 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:55:57,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:57,337 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:55:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:55:57,339 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-27 14:55:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,341 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:57,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:55:57,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-27 14:55:57,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:55:57,342 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-27 14:55:57,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:55:57,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:55:57,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-27 14:55:57,342 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:55:57,443 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:55:57,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:55:57,444 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:55:57,444 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): f53154fc29f582902661898287c317b5/family1 is initiating major compaction (all files) 2023-11-27 14:55:57,444 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:55:57,445 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:55:57,445 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of f53154fc29f582902661898287c317b5/family1 in testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:57,445 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/73d0bbf7e9cb43e3a1561e0e7e56f260, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/44d0a27b9dfe46dcb2a46922bc5cb794, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/8bb35dffb04542408445e0a661ed6843, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/19921ab818304b7ca96849e3388bd514, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/3e099553dd574bd7b9536c0f396d3790] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp, totalSize=26.7 K 2023-11-27 14:55:57,445 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 73d0bbf7e9cb43e3a1561e0e7e56f260, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946676 2023-11-27 14:55:57,446 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 44d0a27b9dfe46dcb2a46922bc5cb794, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:55:57,446 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 8bb35dffb04542408445e0a661ed6843, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950264 2023-11-27 14:55:57,447 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 19921ab818304b7ca96849e3388bd514, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952063 2023-11-27 14:55:57,447 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 3e099553dd574bd7b9536c0f396d3790, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-27 14:55:57,465 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): f53154fc29f582902661898287c317b5#family1#compaction#106 average throughput is 0.28 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:55:57,877 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311274fb50cf26f5540c48c18dfdbb9225791_del 2023-11-27 14:55:57,877 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311274fb50cf26f5540c48c18dfdbb9225791_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311274fb50cf26f5540c48c18dfdbb9225791_del 2023-11-27 14:55:58,692 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c336c9bcd62549f29148b3ad9aa3550a 2023-11-27 14:55:58,693 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/c336c9bcd62549f29148b3ad9aa3550a as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/c336c9bcd62549f29148b3ad9aa3550a 2023-11-27 14:55:58,696 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c336c9bcd62549f29148b3ad9aa3550a 2023-11-27 14:55:58,697 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f53154fc29f582902661898287c317b5/family1 of f53154fc29f582902661898287c317b5 into c336c9bcd62549f29148b3ad9aa3550a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:55:58,698 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:58,698 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:55:58,698 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): f53154fc29f582902661898287c317b5/family2 is initiating major compaction (all files) 2023-11-27 14:55:58,698 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:55:58,698 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:55:58,698 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of f53154fc29f582902661898287c317b5/family2 in testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:55:58,698 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/06188b30d7f149fc989a47c75248d5eb, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/f646c62f8dfd4699bf74c0db3d8ca297, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/be1d45f3a67c4f3eaeea2cd5482139f0, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/384e3e9c79f642ecaf650fef248f348e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/50220b22c7d1438d89f660702790888c] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp, totalSize=25.4 K 2023-11-27 14:55:58,699 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 06188b30d7f149fc989a47c75248d5eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946676 2023-11-27 14:55:58,699 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting f646c62f8dfd4699bf74c0db3d8ca297, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:55:58,699 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting be1d45f3a67c4f3eaeea2cd5482139f0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950264 2023-11-27 14:55:58,700 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 384e3e9c79f642ecaf650fef248f348e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952063 2023-11-27 14:55:58,700 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 50220b22c7d1438d89f660702790888c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-27 14:55:58,711 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): f53154fc29f582902661898287c317b5#family2#compaction#107 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:55:58,753 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/73d0bbf7e9cb43e3a1561e0e7e56f260, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/44d0a27b9dfe46dcb2a46922bc5cb794, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/8bb35dffb04542408445e0a661ed6843, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/19921ab818304b7ca96849e3388bd514, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/3e099553dd574bd7b9536c0f396d3790] to archive 2023-11-27 14:55:58,754 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:58,755 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/73d0bbf7e9cb43e3a1561e0e7e56f260 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/73d0bbf7e9cb43e3a1561e0e7e56f260 2023-11-27 14:55:58,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/44d0a27b9dfe46dcb2a46922bc5cb794 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/44d0a27b9dfe46dcb2a46922bc5cb794 2023-11-27 14:55:58,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/8bb35dffb04542408445e0a661ed6843 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/8bb35dffb04542408445e0a661ed6843 2023-11-27 14:55:58,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/19921ab818304b7ca96849e3388bd514 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/19921ab818304b7ca96849e3388bd514 2023-11-27 14:55:58,760 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/3e099553dd574bd7b9536c0f396d3790 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/3e099553dd574bd7b9536c0f396d3790 2023-11-27 14:55:59,121 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023112735d6b8f154a04b2aa7ac8bf9532fc700_del 2023-11-27 14:55:59,121 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112735d6b8f154a04b2aa7ac8bf9532fc700_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112735d6b8f154a04b2aa7ac8bf9532fc700_del 2023-11-27 14:55:59,936 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2adf075a9d284793bc4ef01eac6f8983 2023-11-27 14:55:59,937 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/2adf075a9d284793bc4ef01eac6f8983 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/2adf075a9d284793bc4ef01eac6f8983 2023-11-27 14:55:59,940 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2adf075a9d284793bc4ef01eac6f8983 2023-11-27 14:55:59,941 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f53154fc29f582902661898287c317b5/family2 of f53154fc29f582902661898287c317b5 into 2adf075a9d284793bc4ef01eac6f8983(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:55:59,942 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:55:59,942 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:55:59,942 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): d2db448818547f8516ae9f9e43b817b6/family1 is initiating major compaction (all files) 2023-11-27 14:55:59,942 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:55:59,942 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:55:59,942 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of d2db448818547f8516ae9f9e43b817b6/family1 in testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:55:59,942 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/9c5c6b42e7aa4057b2d2525439abb851, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/54bff7e3315248878a316aabec8f5fb8, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/83ab5ebd1f744748a511df6081b4ca4e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e77c71cc85014290a851fe41a0dc07ba, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/788c8703e81c49ca807a4a429c2dd649] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp, totalSize=26.7 K 2023-11-27 14:55:59,942 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 9c5c6b42e7aa4057b2d2525439abb851, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946677 2023-11-27 14:55:59,943 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 54bff7e3315248878a316aabec8f5fb8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:55:59,943 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 83ab5ebd1f744748a511df6081b4ca4e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950264 2023-11-27 14:55:59,943 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting e77c71cc85014290a851fe41a0dc07ba, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952064 2023-11-27 14:55:59,944 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 788c8703e81c49ca807a4a429c2dd649, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-27 14:55:59,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/06188b30d7f149fc989a47c75248d5eb, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/f646c62f8dfd4699bf74c0db3d8ca297, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/be1d45f3a67c4f3eaeea2cd5482139f0, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/384e3e9c79f642ecaf650fef248f348e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/50220b22c7d1438d89f660702790888c] to archive 2023-11-27 14:55:59,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:55:59,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/06188b30d7f149fc989a47c75248d5eb to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/06188b30d7f149fc989a47c75248d5eb 2023-11-27 14:55:59,954 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): d2db448818547f8516ae9f9e43b817b6#family1#compaction#108 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:55:59,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/f646c62f8dfd4699bf74c0db3d8ca297 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/f646c62f8dfd4699bf74c0db3d8ca297 2023-11-27 14:55:59,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/be1d45f3a67c4f3eaeea2cd5482139f0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/be1d45f3a67c4f3eaeea2cd5482139f0 2023-11-27 14:55:59,958 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/384e3e9c79f642ecaf650fef248f348e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/384e3e9c79f642ecaf650fef248f348e 2023-11-27 14:55:59,959 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/50220b22c7d1438d89f660702790888c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/50220b22c7d1438d89f660702790888c 2023-11-27 14:56:00,370 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231127b621b95d64564d3bb638016f7d74852a_del 2023-11-27 14:56:00,370 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127b621b95d64564d3bb638016f7d74852a_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127b621b95d64564d3bb638016f7d74852a_del 2023-11-27 14:56:01,185 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cd3d8d0cc74545629a3f31226f7270e4 2023-11-27 14:56:01,185 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/cd3d8d0cc74545629a3f31226f7270e4 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/cd3d8d0cc74545629a3f31226f7270e4 2023-11-27 14:56:01,189 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cd3d8d0cc74545629a3f31226f7270e4 2023-11-27 14:56:01,191 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d2db448818547f8516ae9f9e43b817b6/family1 of d2db448818547f8516ae9f9e43b817b6 into cd3d8d0cc74545629a3f31226f7270e4(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:01,191 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:56:01,191 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:01,191 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): d2db448818547f8516ae9f9e43b817b6/family2 is initiating major compaction (all files) 2023-11-27 14:56:01,191 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:01,191 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:01,192 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of d2db448818547f8516ae9f9e43b817b6/family2 in testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:01,192 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/aacaf93d059f41879ce21e42a7deb376, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/55af4ee02b754f3e80ab86a4399ba221, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/7cade81205724088b52161b761cad33c, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/383ab85ed5fa46a19088c45d22cb1c6f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/8a0a5b2399114ecd83e6fa3a6174c58c] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp, totalSize=25.4 K 2023-11-27 14:56:01,193 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting aacaf93d059f41879ce21e42a7deb376, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946677 2023-11-27 14:56:01,193 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 55af4ee02b754f3e80ab86a4399ba221, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:56:01,194 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 7cade81205724088b52161b761cad33c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950264 2023-11-27 14:56:01,194 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 383ab85ed5fa46a19088c45d22cb1c6f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952064 2023-11-27 14:56:01,195 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 8a0a5b2399114ecd83e6fa3a6174c58c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-27 14:56:01,209 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): d2db448818547f8516ae9f9e43b817b6#family2#compaction#109 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:01,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/9c5c6b42e7aa4057b2d2525439abb851, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/54bff7e3315248878a316aabec8f5fb8, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/83ab5ebd1f744748a511df6081b4ca4e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e77c71cc85014290a851fe41a0dc07ba, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/788c8703e81c49ca807a4a429c2dd649] to archive 2023-11-27 14:56:01,250 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:01,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/9c5c6b42e7aa4057b2d2525439abb851 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/9c5c6b42e7aa4057b2d2525439abb851 2023-11-27 14:56:01,253 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/54bff7e3315248878a316aabec8f5fb8 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/54bff7e3315248878a316aabec8f5fb8 2023-11-27 14:56:01,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/83ab5ebd1f744748a511df6081b4ca4e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/83ab5ebd1f744748a511df6081b4ca4e 2023-11-27 14:56:01,255 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e77c71cc85014290a851fe41a0dc07ba to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e77c71cc85014290a851fe41a0dc07ba 2023-11-27 14:56:01,257 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/788c8703e81c49ca807a4a429c2dd649 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/788c8703e81c49ca807a4a429c2dd649 2023-11-27 14:56:01,618 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231127b703edb1dff04b9db22f640a23386f54_del 2023-11-27 14:56:01,618 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127b703edb1dff04b9db22f640a23386f54_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127b703edb1dff04b9db22f640a23386f54_del 2023-11-27 14:56:02,433 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 053478c9960c406da9c64afca9b098a0 2023-11-27 14:56:02,434 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/053478c9960c406da9c64afca9b098a0 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/053478c9960c406da9c64afca9b098a0 2023-11-27 14:56:02,438 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 053478c9960c406da9c64afca9b098a0 2023-11-27 14:56:02,438 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d2db448818547f8516ae9f9e43b817b6/family2 of d2db448818547f8516ae9f9e43b817b6 into 053478c9960c406da9c64afca9b098a0(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:02,439 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:56:02,439 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:02,439 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): 68bb86a7752ed0247b952754e90231e7/family1 is initiating major compaction (all files) 2023-11-27 14:56:02,439 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:02,439 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:02,439 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of 68bb86a7752ed0247b952754e90231e7/family1 in testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:02,439 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/11dbd4e159484772a2f6b1ba3218b84f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f3476c76367a4094959a000e438862a7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/6cae8ebbf33443cea0a04ac326c2c78e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/08d68f94afe44aa48e17cb61b1ef3c14] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp, totalSize=26.7 K 2023-11-27 14:56:02,440 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 11dbd4e159484772a2f6b1ba3218b84f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946675 2023-11-27 14:56:02,440 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting f3476c76367a4094959a000e438862a7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:56:02,440 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 6cae8ebbf33443cea0a04ac326c2c78e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950263 2023-11-27 14:56:02,441 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting f90fc4bfa7f84ce18764cfcbd9c5c38e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952063 2023-11-27 14:56:02,441 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 08d68f94afe44aa48e17cb61b1ef3c14, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-27 14:56:02,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/aacaf93d059f41879ce21e42a7deb376, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/55af4ee02b754f3e80ab86a4399ba221, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/7cade81205724088b52161b761cad33c, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/383ab85ed5fa46a19088c45d22cb1c6f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/8a0a5b2399114ecd83e6fa3a6174c58c] to archive 2023-11-27 14:56:02,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:02,452 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): 68bb86a7752ed0247b952754e90231e7#family1#compaction#110 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:02,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/aacaf93d059f41879ce21e42a7deb376 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/aacaf93d059f41879ce21e42a7deb376 2023-11-27 14:56:02,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/55af4ee02b754f3e80ab86a4399ba221 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/55af4ee02b754f3e80ab86a4399ba221 2023-11-27 14:56:02,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/7cade81205724088b52161b761cad33c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/7cade81205724088b52161b761cad33c 2023-11-27 14:56:02,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/383ab85ed5fa46a19088c45d22cb1c6f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/383ab85ed5fa46a19088c45d22cb1c6f 2023-11-27 14:56:02,457 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/8a0a5b2399114ecd83e6fa3a6174c58c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/8a0a5b2399114ecd83e6fa3a6174c58c 2023-11-27 14:56:02,860 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311278415e14b96e54b1980973d4957c70558_del 2023-11-27 14:56:02,860 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311278415e14b96e54b1980973d4957c70558_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311278415e14b96e54b1980973d4957c70558_del 2023-11-27 14:56:03,674 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 874a49d1d83f4f3bb5faec70184cfd4f 2023-11-27 14:56:03,675 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/874a49d1d83f4f3bb5faec70184cfd4f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/874a49d1d83f4f3bb5faec70184cfd4f 2023-11-27 14:56:03,679 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 874a49d1d83f4f3bb5faec70184cfd4f 2023-11-27 14:56:03,680 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 68bb86a7752ed0247b952754e90231e7/family1 of 68bb86a7752ed0247b952754e90231e7 into 874a49d1d83f4f3bb5faec70184cfd4f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:03,680 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:56:03,680 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:03,680 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): 68bb86a7752ed0247b952754e90231e7/family2 is initiating major compaction (all files) 2023-11-27 14:56:03,680 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:03,680 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:03,681 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of 68bb86a7752ed0247b952754e90231e7/family2 in testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:03,681 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/ac972ceb881e40c88492f6d98ccfde53, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/62cd5b0e36874070bd39a34ab752b1a6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/3f82744f5963467498262e333eab0bca, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/7362e669758d4b249076960802af5b98, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/49498639cd29446a946d001146acac2f] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp, totalSize=25.4 K 2023-11-27 14:56:03,682 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting ac972ceb881e40c88492f6d98ccfde53, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946675 2023-11-27 14:56:03,682 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 62cd5b0e36874070bd39a34ab752b1a6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:56:03,682 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 3f82744f5963467498262e333eab0bca, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950263 2023-11-27 14:56:03,683 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 7362e669758d4b249076960802af5b98, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952063 2023-11-27 14:56:03,683 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 49498639cd29446a946d001146acac2f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-27 14:56:03,695 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): 68bb86a7752ed0247b952754e90231e7#family2#compaction#111 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:03,755 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/11dbd4e159484772a2f6b1ba3218b84f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f3476c76367a4094959a000e438862a7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/6cae8ebbf33443cea0a04ac326c2c78e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/08d68f94afe44aa48e17cb61b1ef3c14] to archive 2023-11-27 14:56:03,755 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:03,757 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/11dbd4e159484772a2f6b1ba3218b84f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/11dbd4e159484772a2f6b1ba3218b84f 2023-11-27 14:56:03,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f3476c76367a4094959a000e438862a7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f3476c76367a4094959a000e438862a7 2023-11-27 14:56:03,760 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/6cae8ebbf33443cea0a04ac326c2c78e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/6cae8ebbf33443cea0a04ac326c2c78e 2023-11-27 14:56:03,761 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/f90fc4bfa7f84ce18764cfcbd9c5c38e 2023-11-27 14:56:03,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/08d68f94afe44aa48e17cb61b1ef3c14 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/08d68f94afe44aa48e17cb61b1ef3c14 2023-11-27 14:56:04,104 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023112737693c0f43ac441da643eae634155fb6_del 2023-11-27 14:56:04,104 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112737693c0f43ac441da643eae634155fb6_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112737693c0f43ac441da643eae634155fb6_del 2023-11-27 14:56:04,918 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a59e54c657dd4d70b8ebbbd9bb90cdcd 2023-11-27 14:56:04,919 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/a59e54c657dd4d70b8ebbbd9bb90cdcd as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/a59e54c657dd4d70b8ebbbd9bb90cdcd 2023-11-27 14:56:04,922 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a59e54c657dd4d70b8ebbbd9bb90cdcd 2023-11-27 14:56:04,923 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 68bb86a7752ed0247b952754e90231e7/family2 of 68bb86a7752ed0247b952754e90231e7 into a59e54c657dd4d70b8ebbbd9bb90cdcd(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:04,924 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:56:04,928 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:04,932 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:04,937 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:04,938 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:04,944 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-27 14:56:04,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/ac972ceb881e40c88492f6d98ccfde53, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/62cd5b0e36874070bd39a34ab752b1a6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/3f82744f5963467498262e333eab0bca, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/7362e669758d4b249076960802af5b98, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/49498639cd29446a946d001146acac2f] to archive 2023-11-27 14:56:04,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:04,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/ac972ceb881e40c88492f6d98ccfde53 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/ac972ceb881e40c88492f6d98ccfde53 2023-11-27 14:56:04,959 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/62cd5b0e36874070bd39a34ab752b1a6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/62cd5b0e36874070bd39a34ab752b1a6 2023-11-27 14:56:04,960 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/3f82744f5963467498262e333eab0bca to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/3f82744f5963467498262e333eab0bca 2023-11-27 14:56:04,961 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/7362e669758d4b249076960802af5b98 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/7362e669758d4b249076960802af5b98 2023-11-27 14:56:04,963 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/49498639cd29446a946d001146acac2f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/49498639cd29446a946d001146acac2f 2023-11-27 14:56:04,978 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-11-27 14:56:04,981 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311278415e14b96e54b1980973d4957c70558_del 2023-11-27 14:56:04,984 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311274fb50cf26f5540c48c18dfdbb9225791_del 2023-11-27 14:56:04,987 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231127b621b95d64564d3bb638016f7d74852a_del 2023-11-27 14:56:04,987 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:04,988 DEBUG [Listener at localhost/37439] zookeeper.ReadOnlyZKClient(139): Connect 0x41d5435c to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:56:04,992 DEBUG [Listener at localhost/37439] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2136eb7f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:56:04,992 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:04,993 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:04,993 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:05,474 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127f17b46bf054747fca37b924f85cc725e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127f17b46bf054747fca37b924f85cc725e 2023-11-27 14:56:05,475 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:05,478 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112703d72353645441beb4497cbd9d5b647b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112703d72353645441beb4497cbd9d5b647b 2023-11-27 14:56:05,478 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:05,481 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127adb5bf5edef24c37a4a5d0ccd70bc643 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127adb5bf5edef24c37a4a5d0ccd70bc643 2023-11-27 14:56:05,481 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:05,489 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:56:05,493 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43694, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:56:05,503 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:56:05,506 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38408, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:56:05,518 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127/family1/3a0eeebad088499d84260cc0f50e7f40 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-11-27 14:56:05,518 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127/family1/ec728461a9764c9fbf195a02dabf0d6b first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-11-27 14:56:05,518 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127/family1/8ea77618414141c1a6ac8b13f18ab67c first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-11-27 14:56:05,520 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127/family1/3a0eeebad088499d84260cc0f50e7f40}] 2023-11-27 14:56:05,521 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127/family1/ec728461a9764c9fbf195a02dabf0d6b}] 2023-11-27 14:56:05,521 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127/family1/8ea77618414141c1a6ac8b13f18ab67c}] 2023-11-27 14:56:05,529 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127/family1/ec728461a9764c9fbf195a02dabf0d6b for inclusion in d2db448818547f8516ae9f9e43b817b6/family1 2023-11-27 14:56:05,530 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127/family1/3a0eeebad088499d84260cc0f50e7f40 for inclusion in 68bb86a7752ed0247b952754e90231e7/family1 2023-11-27 14:56:05,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127/family1/8ea77618414141c1a6ac8b13f18ab67c for inclusion in f53154fc29f582902661898287c317b5/family1 2023-11-27 14:56:05,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-11-27 14:56:05,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-11-27 14:56:05,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-27 14:56:05,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last=1 2023-11-27 14:56:05,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-11-27 14:56:05,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:56:05,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-27 14:56:05,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:56:05,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:56:05,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127/family1/3a0eeebad088499d84260cc0f50e7f40 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__275kr369d51jnksdlv9k0qh5c399n82dibn9bqoei94a18ujpljq6l9fo6algpt6/family1/3a0eeebad088499d84260cc0f50e7f40 2023-11-27 14:56:05,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127/family1/8ea77618414141c1a6ac8b13f18ab67c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__9g27gm1jqtnuaeta3khq8red7sh2cvlu25a62iuacuo9h98fv1rf25fuc689g0fe/family1/8ea77618414141c1a6ac8b13f18ab67c 2023-11-27 14:56:05,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127/family1/ec728461a9764c9fbf195a02dabf0d6b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__rv5ladn32uad24gpuqkcq07j8j9b9nagfdnpldb5h3tmb7ng1il1klpuqma0erv6/family1/ec728461a9764c9fbf195a02dabf0d6b 2023-11-27 14:56:05,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__275kr369d51jnksdlv9k0qh5c399n82dibn9bqoei94a18ujpljq6l9fo6algpt6/family1/3a0eeebad088499d84260cc0f50e7f40 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/952ec37ad54541f6bf9a657c0dfd4de2_SeqId_29_ 2023-11-27 14:56:05,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__9g27gm1jqtnuaeta3khq8red7sh2cvlu25a62iuacuo9h98fv1rf25fuc689g0fe/family1/8ea77618414141c1a6ac8b13f18ab67c as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/2c0581ef709c45d4b20d8ac3f2f2d756_SeqId_29_ 2023-11-27 14:56:05,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__rv5ladn32uad24gpuqkcq07j8j9b9nagfdnpldb5h3tmb7ng1il1klpuqma0erv6/family1/ec728461a9764c9fbf195a02dabf0d6b as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e588d716c5774d34a364de66f6bcc4b7_SeqId_29_ 2023-11-27 14:56:05,538 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__275kr369d51jnksdlv9k0qh5c399n82dibn9bqoei94a18ujpljq6l9fo6algpt6/family1/3a0eeebad088499d84260cc0f50e7f40 into 68bb86a7752ed0247b952754e90231e7/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/952ec37ad54541f6bf9a657c0dfd4de2_SeqId_29_ - updating store file list. 2023-11-27 14:56:05,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__9g27gm1jqtnuaeta3khq8red7sh2cvlu25a62iuacuo9h98fv1rf25fuc689g0fe/family1/8ea77618414141c1a6ac8b13f18ab67c into f53154fc29f582902661898287c317b5/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/2c0581ef709c45d4b20d8ac3f2f2d756_SeqId_29_ - updating store file list. 2023-11-27 14:56:05,539 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__rv5ladn32uad24gpuqkcq07j8j9b9nagfdnpldb5h3tmb7ng1il1klpuqma0erv6/family1/ec728461a9764c9fbf195a02dabf0d6b into d2db448818547f8516ae9f9e43b817b6/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e588d716c5774d34a364de66f6bcc4b7_SeqId_29_ - updating store file list. 2023-11-27 14:56:05,542 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e588d716c5774d34a364de66f6bcc4b7_SeqId_29_ into d2db448818547f8516ae9f9e43b817b6/family1 2023-11-27 14:56:05,542 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/952ec37ad54541f6bf9a657c0dfd4de2_SeqId_29_ into 68bb86a7752ed0247b952754e90231e7/family1 2023-11-27 14:56:05,542 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__rv5ladn32uad24gpuqkcq07j8j9b9nagfdnpldb5h3tmb7ng1il1klpuqma0erv6/family1/ec728461a9764c9fbf195a02dabf0d6b into d2db448818547f8516ae9f9e43b817b6/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e588d716c5774d34a364de66f6bcc4b7_SeqId_29_) 2023-11-27 14:56:05,543 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__275kr369d51jnksdlv9k0qh5c399n82dibn9bqoei94a18ujpljq6l9fo6algpt6/family1/3a0eeebad088499d84260cc0f50e7f40 into 68bb86a7752ed0247b952754e90231e7/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/952ec37ad54541f6bf9a657c0dfd4de2_SeqId_29_) 2023-11-27 14:56:05,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__rv5ladn32uad24gpuqkcq07j8j9b9nagfdnpldb5h3tmb7ng1il1klpuqma0erv6/family1/ec728461a9764c9fbf195a02dabf0d6b 2023-11-27 14:56:05,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__275kr369d51jnksdlv9k0qh5c399n82dibn9bqoei94a18ujpljq6l9fo6algpt6/family1/3a0eeebad088499d84260cc0f50e7f40 2023-11-27 14:56:05,545 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:05,545 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:05,546 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:05,546 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:05,548 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311273f6fd9351e9842df8dddeb3f1bd00e91 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311273f6fd9351e9842df8dddeb3f1bd00e91 2023-11-27 14:56:05,548 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112743713d96bf2d4fa7878b42c12e1a7fe5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023112743713d96bf2d4fa7878b42c12e1a7fe5 2023-11-27 14:56:05,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/2c0581ef709c45d4b20d8ac3f2f2d756_SeqId_29_ into f53154fc29f582902661898287c317b5/family1 2023-11-27 14:56:05,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__9g27gm1jqtnuaeta3khq8red7sh2cvlu25a62iuacuo9h98fv1rf25fuc689g0fe/family1/8ea77618414141c1a6ac8b13f18ab67c into f53154fc29f582902661898287c317b5/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/2c0581ef709c45d4b20d8ac3f2f2d756_SeqId_29_) 2023-11-27 14:56:05,549 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311274a4801ad97a94826a437fba4aaced078 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311274a4801ad97a94826a437fba4aaced078 2023-11-27 14:56:05,549 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311278bd748d50ec449b591ab4997d2b6bb2a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311278bd748d50ec449b591ab4997d2b6bb2a 2023-11-27 14:56:05,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__9g27gm1jqtnuaeta3khq8red7sh2cvlu25a62iuacuo9h98fv1rf25fuc689g0fe/family1/8ea77618414141c1a6ac8b13f18ab67c 2023-11-27 14:56:05,551 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127ca141289f66b4b37a06eaec5c63297ae to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127ca141289f66b4b37a06eaec5c63297ae 2023-11-27 14:56:05,551 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311279f6d46bf55ff44f8a38c99407a29d75f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311279f6d46bf55ff44f8a38c99407a29d75f 2023-11-27 14:56:05,551 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:05,552 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:05,552 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127deae02fa52974d90841c7ec64ee944df to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127deae02fa52974d90841c7ec64ee944df 2023-11-27 14:56:05,552 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-27 14:56:05,552 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127' 2023-11-27 14:56:05,552 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127fedadc64d2504dccb03740338659f0c6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127fedadc64d2504dccb03740338659f0c6 2023-11-27 14:56:05,553 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-27 14:56:05,553 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127' 2023-11-27 14:56:05,553 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:05,553 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:05,554 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311276c172b1bef3842fea773178819944ccb to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311276c172b1bef3842fea773178819944ccb 2023-11-27 14:56:05,555 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ae5335fb8ba24277a60870ae7f9a4267 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ae5335fb8ba24277a60870ae7f9a4267 2023-11-27 14:56:05,556 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ecf30d17888a4c878b4e64030723965f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127ecf30d17888a4c878b4e64030723965f 2023-11-27 14:56:05,557 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127f4b90f9713bc44fc9fb9c60ce591c905 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231127f4b90f9713bc44fc9fb9c60ce591c905 2023-11-27 14:56:05,557 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-27 14:56:05,557 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127' 2023-11-27 14:56:05,557 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:05,557 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:05,557 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:05,558 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(367): [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311278415e14b96e54b1980973d4957c70558_del] 2023-11-27 14:56:05,558 DEBUG [Listener at localhost/37439] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:05,559 DEBUG [Listener at localhost/37439] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311278415e14b96e54b1980973d4957c70558_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311278415e14b96e54b1980973d4957c70558_del 2023-11-27 14:56:05,559 INFO [Listener at localhost/37439] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-27 14:56:05,559 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(367): [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311274fb50cf26f5540c48c18dfdbb9225791_del] 2023-11-27 14:56:05,560 DEBUG [Listener at localhost/37439] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:05,561 DEBUG [Listener at localhost/37439] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311274fb50cf26f5540c48c18dfdbb9225791_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311274fb50cf26f5540c48c18dfdbb9225791_del 2023-11-27 14:56:05,561 INFO [Listener at localhost/37439] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-27 14:56:05,561 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(367): [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127b621b95d64564d3bb638016f7d74852a_del] 2023-11-27 14:56:05,561 DEBUG [Listener at localhost/37439] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:05,563 DEBUG [Listener at localhost/37439] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127b621b95d64564d3bb638016f7d74852a_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127b621b95d64564d3bb638016f7d74852a_del 2023-11-27 14:56:05,563 INFO [Listener at localhost/37439] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-27 14:56:05,566 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:05,569 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:05,572 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:05,573 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:05,578 INFO [Listener at localhost/37439] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-11-27 14:56:05,578 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-11-27 14:56:05,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-11-27 14:56:05,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-27 14:56:05,581 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096965581"}]},"ts":"1701096965581"} 2023-11-27 14:56:05,582 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-11-27 14:56:05,585 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-11-27 14:56:05,586 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, UNASSIGN}] 2023-11-27 14:56:05,587 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, UNASSIGN 2023-11-27 14:56:05,587 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, UNASSIGN 2023-11-27 14:56:05,587 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, UNASSIGN 2023-11-27 14:56:05,588 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f53154fc29f582902661898287c317b5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:05,588 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=d2db448818547f8516ae9f9e43b817b6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:05,588 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=68bb86a7752ed0247b952754e90231e7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:05,588 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096965588"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096965588"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096965588"}]},"ts":"1701096965588"} 2023-11-27 14:56:05,588 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701096965588"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096965588"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096965588"}]},"ts":"1701096965588"} 2023-11-27 14:56:05,588 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096965588"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096965588"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096965588"}]},"ts":"1701096965588"} 2023-11-27 14:56:05,589 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure d2db448818547f8516ae9f9e43b817b6, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:05,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure f53154fc29f582902661898287c317b5, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:05,590 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure 68bb86a7752ed0247b952754e90231e7, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:05,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-27 14:56:05,715 ERROR [ProcedureDispatcherTimeoutThread] procedure2.RemoteProcedureDispatcher$TimeoutExecutorThread(324): DelayQueue for RemoteProcedureDispatcher is not empty when timed waiting elapsed. If this is repeated consistently, it means no element is getting expired from the queue and it might freeze the system. Queue: [containedObject=jenkins-hbase4.apache.org,42669,1701096850360, timeout=1701096965740, delay=25, operations=[pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure 68bb86a7752ed0247b952754e90231e7, server=jenkins-hbase4.apache.org,42669,1701096850360, pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure f53154fc29f582902661898287c317b5, server=jenkins-hbase4.apache.org,42669,1701096850360, pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure d2db448818547f8516ae9f9e43b817b6, server=jenkins-hbase4.apache.org,42669,1701096850360]] 2023-11-27 14:56:05,740 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:05,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:05,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 68bb86a7752ed0247b952754e90231e7, disabling compactions & flushes 2023-11-27 14:56:05,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:05,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:05,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. after waiting 0 ms 2023-11-27 14:56:05,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:05,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-27 14:56:05,746 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:05,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:56:05,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:05,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close f53154fc29f582902661898287c317b5 2023-11-27 14:56:05,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f53154fc29f582902661898287c317b5, disabling compactions & flushes 2023-11-27 14:56:05,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:05,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:05,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. after waiting 0 ms 2023-11-27 14:56:05,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:05,748 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=68bb86a7752ed0247b952754e90231e7, regionState=CLOSED 2023-11-27 14:56:05,748 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096965748"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096965748"}]},"ts":"1701096965748"} 2023-11-27 14:56:05,751 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=94, resume processing ppid=89 2023-11-27 14:56:05,751 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure 68bb86a7752ed0247b952754e90231e7, server=jenkins-hbase4.apache.org,42669,1701096850360 in 159 msec 2023-11-27 14:56:05,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-27 14:56:05,753 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, UNASSIGN in 165 msec 2023-11-27 14:56:05,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:05,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:56:05,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed f53154fc29f582902661898287c317b5 2023-11-27 14:56:05,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:05,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d2db448818547f8516ae9f9e43b817b6, disabling compactions & flushes 2023-11-27 14:56:05,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:05,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:05,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. after waiting 0 ms 2023-11-27 14:56:05,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:05,755 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f53154fc29f582902661898287c317b5, regionState=CLOSED 2023-11-27 14:56:05,755 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701096965755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096965755"}]},"ts":"1701096965755"} 2023-11-27 14:56:05,758 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=93, resume processing ppid=90 2023-11-27 14:56:05,759 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure f53154fc29f582902661898287c317b5, server=jenkins-hbase4.apache.org,42669,1701096850360 in 168 msec 2023-11-27 14:56:05,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, UNASSIGN in 172 msec 2023-11-27 14:56:05,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-27 14:56:05,764 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:05,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:56:05,765 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:05,766 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=d2db448818547f8516ae9f9e43b817b6, regionState=CLOSED 2023-11-27 14:56:05,766 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096965766"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096965766"}]},"ts":"1701096965766"} 2023-11-27 14:56:05,768 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=92, resume processing ppid=91 2023-11-27 14:56:05,768 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure d2db448818547f8516ae9f9e43b817b6, server=jenkins-hbase4.apache.org,42669,1701096850360 in 178 msec 2023-11-27 14:56:05,770 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=91, resume processing ppid=88 2023-11-27 14:56:05,770 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, UNASSIGN in 182 msec 2023-11-27 14:56:05,770 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096965770"}]},"ts":"1701096965770"} 2023-11-27 14:56:05,771 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-11-27 14:56:05,773 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-11-27 14:56:05,775 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 195 msec 2023-11-27 14:56:05,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-27 14:56:05,883 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-11-27 14:56:05,895 INFO [Listener at localhost/37439] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1701096953866-1701096965895 2023-11-27 14:56:05,896 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-27 14:56:05,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1701096965896 (current time:1701096965896). 2023-11-27 14:56:05,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-11-27 14:56:05,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] zookeeper.ReadOnlyZKClient(139): Connect 0x33c68a4f to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:56:05,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@fc1b0f3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:56:05,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:56:05,904 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43704, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:56:05,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33c68a4f to 127.0.0.1:61098 2023-11-27 14:56:05,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:05,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-11-27 14:56:05,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(695): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-27 14:56:05,908 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot, current working dir path: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701096953866-1701096965895 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotDisabledTable(SnapshotManager.java:541) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:698) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-11-27 14:56:05,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-27 14:56:05,911 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-27 14:56:05,911 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1701097565911 2023-11-27 14:56:05,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(700): Started snapshot: { ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-27 14:56:05,911 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1701096953866-1701096965895 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-11-27 14:56:05,912 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:05,912 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-11-27 14:56:06,012 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:56:06,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:56:06,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-27 14:56:06,013 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-11-27 14:56:06,213 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:56:06,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:56:06,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-27 14:56:06,215 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-11-27 14:56:06,318 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-11-27 14:56:06,320 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-27 14:56:06,320 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-27 14:56:06,320 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-27 14:56:06,320 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-27 14:56:06,321 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-27 14:56:06,320 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-27 14:56:06,320 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-27 14:56:06,320 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/2c0581ef709c45d4b20d8ac3f2f2d756_SeqId_29_, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/c336c9bcd62549f29148b3ad9aa3550a] hfiles 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/874a49d1d83f4f3bb5faec70184cfd4f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/952ec37ad54541f6bf9a657c0dfd4de2_SeqId_29_] hfiles 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/2c0581ef709c45d4b20d8ac3f2f2d756_SeqId_29_ 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/874a49d1d83f4f3bb5faec70184cfd4f 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/c336c9bcd62549f29148b3ad9aa3550a 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/cd3d8d0cc74545629a3f31226f7270e4, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e588d716c5774d34a364de66f6bcc4b7_SeqId_29_] hfiles 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/952ec37ad54541f6bf9a657c0dfd4de2_SeqId_29_ 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/cd3d8d0cc74545629a3f31226f7270e4 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e588d716c5774d34a364de66f6bcc4b7_SeqId_29_ 2023-11-27 14:56:06,324 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112703d72353645441beb4497cbd9d5b647b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127adb5bf5edef24c37a4a5d0ccd70bc643, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127f17b46bf054747fca37b924f85cc725e] hfiles 2023-11-27 14:56:06,325 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112703d72353645441beb4497cbd9d5b647b 2023-11-27 14:56:06,325 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127adb5bf5edef24c37a4a5d0ccd70bc643 2023-11-27 14:56:06,325 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127f17b46bf054747fca37b924f85cc725e 2023-11-27 14:56:06,332 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/a59e54c657dd4d70b8ebbbd9bb90cdcd] hfiles 2023-11-27 14:56:06,332 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/a59e54c657dd4d70b8ebbbd9bb90cdcd 2023-11-27 14:56:06,332 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/2adf075a9d284793bc4ef01eac6f8983] hfiles 2023-11-27 14:56:06,332 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/2adf075a9d284793bc4ef01eac6f8983 2023-11-27 14:56:06,332 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/053478c9960c406da9c64afca9b098a0] hfiles 2023-11-27 14:56:06,332 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/053478c9960c406da9c64afca9b098a0 2023-11-27 14:56:06,339 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112725e6f6dea04343688bbff668571aadd3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112735d6b8f154a04b2aa7ac8bf9532fc700_del, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112736f5012b8d4b4c318a8dcd007b924aca, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311276b40aff22efd47b288a2aca766bd3f97, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231127f50035775c9a41d3ae3c6c36f321bc32, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127018167e6893d4b1d9cdc8a9dcec1aa7d, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311274996160064294a1195a80e166a6d7d89, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311275dfa46da3cbe4203997d262ff2da49ac, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127728aca12e81d412bb59d2d32c500c398, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127b703edb1dff04b9db22f640a23386f54_del, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112703c1690fa86b4b6c800c5d46c68e7a7c, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311272d64f381bd574ed7a0b3846e5a11f70a, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112737693c0f43ac441da643eae634155fb6_del, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311277de6939e72d543e5abb705afb4a0f052, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231127f1126d91cd6a4b99a8e2daddef1349a0] hfiles 2023-11-27 14:56:06,339 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112725e6f6dea04343688bbff668571aadd3 2023-11-27 14:56:06,339 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112735d6b8f154a04b2aa7ac8bf9532fc700_del 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112736f5012b8d4b4c318a8dcd007b924aca 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311276b40aff22efd47b288a2aca766bd3f97 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231127f50035775c9a41d3ae3c6c36f321bc32 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127018167e6893d4b1d9cdc8a9dcec1aa7d 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311274996160064294a1195a80e166a6d7d89 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311275dfa46da3cbe4203997d262ff2da49ac 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127728aca12e81d412bb59d2d32c500c398 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127b703edb1dff04b9db22f640a23386f54_del 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112703c1690fa86b4b6c800c5d46c68e7a7c 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311272d64f381bd574ed7a0b3846e5a11f70a 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112737693c0f43ac441da643eae634155fb6_del 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311277de6939e72d543e5abb705afb4a0f052 2023-11-27 14:56:06,340 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231127f1126d91cd6a4b99a8e2daddef1349a0 2023-11-27 14:56:06,515 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:56:06,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:56:06,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-27 14:56:06,516 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-11-27 14:56:06,746 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-11-27 14:56:06,746 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1701096953866-1701096965895 2023-11-27 14:56:06,746 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701096953866-1701096965895 2023-11-27 14:56:07,016 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:56:07,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:56:07,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-27 14:56:07,017 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-11-27 14:56:07,173 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701096953866-1701096965895 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/hbase-failsafe-snaptb-1701096953866-1701096965895 2023-11-27 14:56:07,173 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1701096953866-1701096965895 of table testCompactionWithHFileLink completed 2023-11-27 14:56:07,173 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701096953866-1701096965895 2023-11-27 14:56:07,174 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1701096953866-1701096965895 2023-11-27 14:56:07,174 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1701096953866-1701096965895 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1701096965912Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1701096966318 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1701096953866-1701096965895 at 1701096966746 (+428 ms)Loading Region manifests for hbase-failsafe-snaptb-1701096953866-1701096965895 at 1701096966746Writing data manifest for hbase-failsafe-snaptb-1701096953866-1701096965895 at 1701096966749 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1701096953866-1701096965895 at 1701096967155 (+406 ms)Snapshot hbase-failsafe-snaptb-1701096953866-1701096965895 of table testCompactionWithHFileLink completed at 1701096967173 (+18 ms) 2023-11-27 14:56:07,175 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-27 14:56:07,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.2660 sec 2023-11-27 14:56:08,018 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-27 14:56:08,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-27 14:56:08,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(403): Snapshot '{ ss=hbase-failsafe-snaptb-1701096953866-1701096965895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-11-27 14:56:08,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1701096953866" table: "testCompactionWithHFileLink" creation_time: 1701096953876 type: FLUSH version: 2 ttl: 0 ) 2023-11-27 14:56:08,065 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(916): Restore snapshot=snaptb-1701096953866 as table=testCompactionWithHFileLink 2023-11-27 14:56:08,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-27 14:56:08,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-27 14:56:08,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-27 14:56:08,475 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-11-27 14:56:08,476 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-11-27 14:56:08,476 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-11-27 14:56:08,477 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-27 14:56:08,481 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1701096953866" table: "testCompactionWithHFileLink" creation_time: 1701096953876 type: FLUSH version: 2 ttl: 0 2023-11-27 14:56:08,482 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink 2023-11-27 14:56:08,487 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-11-27 14:56:08,490 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127adb5bf5edef24c37a4a5d0ccd70bc643 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231127adb5bf5edef24c37a4a5d0ccd70bc643 2023-11-27 14:56:08,491 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127f17b46bf054747fca37b924f85cc725e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127f17b46bf054747fca37b924f85cc725e 2023-11-27 14:56:08,492 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112703d72353645441beb4497cbd9d5b647b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023112703d72353645441beb4497cbd9d5b647b 2023-11-27 14:56:08,492 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202311276c172b1bef3842fea773178819944ccb of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,496 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231127ae5335fb8ba24277a60870ae7f9a4267 of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,499 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231127ecf30d17888a4c878b4e64030723965f of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,502 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231127f4b90f9713bc44fc9fb9c60ce591c905 of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,505 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202311273f6fd9351e9842df8dddeb3f1bd00e91 of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,507 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202311274a4801ad97a94826a437fba4aaced078 of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,510 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202311279f6d46bf55ff44f8a38c99407a29d75f of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,513 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20231127fedadc64d2504dccb03740338659f0c6 of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,516 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023112743713d96bf2d4fa7878b42c12e1a7fe5 of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,518 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202311278bd748d50ec449b591ab4997d2b6bb2a of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,521 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20231127ca141289f66b4b37a06eaec5c63297ae of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,524 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20231127deae02fa52974d90841c7ec64ee944df of snapshot=snaptb-1701096953866 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-27 14:56:08,530 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112735d6b8f154a04b2aa7ac8bf9532fc700_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023112735d6b8f154a04b2aa7ac8bf9532fc700_del 2023-11-27 14:56:08,532 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112737693c0f43ac441da643eae634155fb6_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023112737693c0f43ac441da643eae634155fb6_del 2023-11-27 14:56:08,535 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127b703edb1dff04b9db22f640a23386f54_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231127b703edb1dff04b9db22f640a23386f54_del 2023-11-27 14:56:08,535 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:08,535 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:08,535 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: f53154fc29f582902661898287c317b5 2023-11-27 14:56:08,546 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/2c0581ef709c45d4b20d8ac3f2f2d756_SeqId_29_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/2c0581ef709c45d4b20d8ac3f2f2d756_SeqId_29_ 2023-11-27 14:56:08,546 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e588d716c5774d34a364de66f6bcc4b7_SeqId_29_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/e588d716c5774d34a364de66f6bcc4b7_SeqId_29_ 2023-11-27 14:56:08,547 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/952ec37ad54541f6bf9a657c0dfd4de2_SeqId_29_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/952ec37ad54541f6bf9a657c0dfd4de2_SeqId_29_ 2023-11-27 14:56:08,548 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/c336c9bcd62549f29148b3ad9aa3550a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/c336c9bcd62549f29148b3ad9aa3550a 2023-11-27 14:56:08,548 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 73d0bbf7e9cb43e3a1561e0e7e56f260 of snapshot=snaptb-1701096953866 to region=f53154fc29f582902661898287c317b5 table=testCompactionWithHFileLink 2023-11-27 14:56:08,548 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/cd3d8d0cc74545629a3f31226f7270e4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/cd3d8d0cc74545629a3f31226f7270e4 2023-11-27 14:56:08,548 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9c5c6b42e7aa4057b2d2525439abb851 of snapshot=snaptb-1701096953866 to region=d2db448818547f8516ae9f9e43b817b6 table=testCompactionWithHFileLink 2023-11-27 14:56:08,549 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/874a49d1d83f4f3bb5faec70184cfd4f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/874a49d1d83f4f3bb5faec70184cfd4f 2023-11-27 14:56:08,549 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 11dbd4e159484772a2f6b1ba3218b84f of snapshot=snaptb-1701096953866 to region=68bb86a7752ed0247b952754e90231e7 table=testCompactionWithHFileLink 2023-11-27 14:56:08,556 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 54bff7e3315248878a316aabec8f5fb8 of snapshot=snaptb-1701096953866 to region=d2db448818547f8516ae9f9e43b817b6 table=testCompactionWithHFileLink 2023-11-27 14:56:08,559 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 44d0a27b9dfe46dcb2a46922bc5cb794 of snapshot=snaptb-1701096953866 to region=f53154fc29f582902661898287c317b5 table=testCompactionWithHFileLink 2023-11-27 14:56:08,560 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f3476c76367a4094959a000e438862a7 of snapshot=snaptb-1701096953866 to region=68bb86a7752ed0247b952754e90231e7 table=testCompactionWithHFileLink 2023-11-27 14:56:08,563 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 83ab5ebd1f744748a511df6081b4ca4e of snapshot=snaptb-1701096953866 to region=d2db448818547f8516ae9f9e43b817b6 table=testCompactionWithHFileLink 2023-11-27 14:56:08,564 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8bb35dffb04542408445e0a661ed6843 of snapshot=snaptb-1701096953866 to region=f53154fc29f582902661898287c317b5 table=testCompactionWithHFileLink 2023-11-27 14:56:08,568 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e77c71cc85014290a851fe41a0dc07ba of snapshot=snaptb-1701096953866 to region=d2db448818547f8516ae9f9e43b817b6 table=testCompactionWithHFileLink 2023-11-27 14:56:08,569 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6cae8ebbf33443cea0a04ac326c2c78e of snapshot=snaptb-1701096953866 to region=68bb86a7752ed0247b952754e90231e7 table=testCompactionWithHFileLink 2023-11-27 14:56:08,569 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 19921ab818304b7ca96849e3388bd514 of snapshot=snaptb-1701096953866 to region=f53154fc29f582902661898287c317b5 table=testCompactionWithHFileLink 2023-11-27 14:56:08,573 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f90fc4bfa7f84ce18764cfcbd9c5c38e of snapshot=snaptb-1701096953866 to region=68bb86a7752ed0247b952754e90231e7 table=testCompactionWithHFileLink 2023-11-27 14:56:08,575 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/053478c9960c406da9c64afca9b098a0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/053478c9960c406da9c64afca9b098a0 2023-11-27 14:56:08,575 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink aacaf93d059f41879ce21e42a7deb376 of snapshot=snaptb-1701096953866 to region=d2db448818547f8516ae9f9e43b817b6 table=testCompactionWithHFileLink 2023-11-27 14:56:08,575 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/2adf075a9d284793bc4ef01eac6f8983 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/2adf075a9d284793bc4ef01eac6f8983 2023-11-27 14:56:08,575 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 06188b30d7f149fc989a47c75248d5eb of snapshot=snaptb-1701096953866 to region=f53154fc29f582902661898287c317b5 table=testCompactionWithHFileLink 2023-11-27 14:56:08,581 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 55af4ee02b754f3e80ab86a4399ba221 of snapshot=snaptb-1701096953866 to region=d2db448818547f8516ae9f9e43b817b6 table=testCompactionWithHFileLink 2023-11-27 14:56:08,582 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/a59e54c657dd4d70b8ebbbd9bb90cdcd to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/a59e54c657dd4d70b8ebbbd9bb90cdcd 2023-11-27 14:56:08,582 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ac972ceb881e40c88492f6d98ccfde53 of snapshot=snaptb-1701096953866 to region=68bb86a7752ed0247b952754e90231e7 table=testCompactionWithHFileLink 2023-11-27 14:56:08,593 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7cade81205724088b52161b761cad33c of snapshot=snaptb-1701096953866 to region=d2db448818547f8516ae9f9e43b817b6 table=testCompactionWithHFileLink 2023-11-27 14:56:08,595 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 62cd5b0e36874070bd39a34ab752b1a6 of snapshot=snaptb-1701096953866 to region=68bb86a7752ed0247b952754e90231e7 table=testCompactionWithHFileLink 2023-11-27 14:56:08,595 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f646c62f8dfd4699bf74c0db3d8ca297 of snapshot=snaptb-1701096953866 to region=f53154fc29f582902661898287c317b5 table=testCompactionWithHFileLink 2023-11-27 14:56:08,597 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 383ab85ed5fa46a19088c45d22cb1c6f of snapshot=snaptb-1701096953866 to region=d2db448818547f8516ae9f9e43b817b6 table=testCompactionWithHFileLink 2023-11-27 14:56:08,603 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3f82744f5963467498262e333eab0bca of snapshot=snaptb-1701096953866 to region=68bb86a7752ed0247b952754e90231e7 table=testCompactionWithHFileLink 2023-11-27 14:56:08,604 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink be1d45f3a67c4f3eaeea2cd5482139f0 of snapshot=snaptb-1701096953866 to region=f53154fc29f582902661898287c317b5 table=testCompactionWithHFileLink 2023-11-27 14:56:08,608 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7362e669758d4b249076960802af5b98 of snapshot=snaptb-1701096953866 to region=68bb86a7752ed0247b952754e90231e7 table=testCompactionWithHFileLink 2023-11-27 14:56:08,608 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 384e3e9c79f642ecaf650fef248f348e of snapshot=snaptb-1701096953866 to region=f53154fc29f582902661898287c317b5 table=testCompactionWithHFileLink 2023-11-27 14:56:08,612 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1701096953866" table: "testCompactionWithHFileLink" creation_time: 1701096953876 type: FLUSH version: 2 ttl: 0 2023-11-27 14:56:08,613 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1701096968613"}]},"ts":"9223372036854775807"} 2023-11-27 14:56:08,613 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1701096968613"}]},"ts":"9223372036854775807"} 2023-11-27 14:56:08,613 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1701096968613"}]},"ts":"9223372036854775807"} 2023-11-27 14:56:08,615 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-27 14:56:08,615 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 68bb86a7752ed0247b952754e90231e7, NAME => 'testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => d2db448818547f8516ae9f9e43b817b6, NAME => 'testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => f53154fc29f582902661898287c317b5, NAME => 'testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.', STARTKEY => '1', ENDKEY => '2'}] 2023-11-27 14:56:08,615 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096968614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096968614"}]},"ts":"1701096968614"} 2023-11-27 14:56:08,615 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096968614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096968614"}]},"ts":"1701096968614"} 2023-11-27 14:56:08,615 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701096968614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096968614"}]},"ts":"1701096968614"} 2023-11-27 14:56:08,617 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-27 14:56:08,617 INFO [PEWorker-3] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-11-27 14:56:08,617 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 68bb86a7752ed0247b952754e90231e7, NAME => 'testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => d2db448818547f8516ae9f9e43b817b6, NAME => 'testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => f53154fc29f582902661898287c317b5, NAME => 'testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.', STARTKEY => '1', ENDKEY => '2'}] 2023-11-27 14:56:08,617 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1701096953866 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-11-27 14:56:08,619 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1701096953866" table: "testCompactionWithHFileLink" creation_time: 1701096953876 type: FLUSH version: 2 ttl: 0 ) in 563 msec 2023-11-27 14:56:08,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-27 14:56:08,671 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-11-27 14:56:08,672 INFO [Listener at localhost/37439] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1701096953866-1701096965895 2023-11-27 14:56:08,677 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(751): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1701096953866-1701096965895" 2023-11-27 14:56:08,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] snapshot.SnapshotManager(335): Deleting snapshot: hbase-failsafe-snaptb-1701096953866-1701096965895 2023-11-27 14:56:08,682 INFO [Listener at localhost/37439] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-11-27 14:56:08,688 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-11-27 14:56:08,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-11-27 14:56:08,698 INFO [PEWorker-5] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-11-27 14:56:08,698 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096968698"}]},"ts":"1701096968698"} 2023-11-27 14:56:08,699 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-11-27 14:56:08,703 DEBUG [PEWorker-5] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-11-27 14:56:08,703 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, ASSIGN}] 2023-11-27 14:56:08,705 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, ASSIGN 2023-11-27 14:56:08,705 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, ASSIGN 2023-11-27 14:56:08,705 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, ASSIGN 2023-11-27 14:56:08,706 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-27 14:56:08,706 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-27 14:56:08,707 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-27 14:56:08,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-27 14:56:08,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-27 14:56:08,858 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=d2db448818547f8516ae9f9e43b817b6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:08,858 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=68bb86a7752ed0247b952754e90231e7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:08,858 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096968858"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096968858"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096968858"}]},"ts":"1701096968858"} 2023-11-27 14:56:08,858 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=f53154fc29f582902661898287c317b5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:08,858 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096968858"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096968858"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096968858"}]},"ts":"1701096968858"} 2023-11-27 14:56:08,858 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701096968858"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096968858"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096968858"}]},"ts":"1701096968858"} 2023-11-27 14:56:08,860 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure d2db448818547f8516ae9f9e43b817b6, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:08,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure 68bb86a7752ed0247b952754e90231e7, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:08,861 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure f53154fc29f582902661898287c317b5, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:09,011 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:09,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:09,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f53154fc29f582902661898287c317b5, NAME => 'testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.', STARTKEY => '1', ENDKEY => '2'} 2023-11-27 14:56:09,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink f53154fc29f582902661898287c317b5 2023-11-27 14:56:09,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:09,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f53154fc29f582902661898287c317b5 2023-11-27 14:56:09,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f53154fc29f582902661898287c317b5 2023-11-27 14:56:09,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-27 14:56:09,016 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f53154fc29f582902661898287c317b5 2023-11-27 14:56:09,017 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1 2023-11-27 14:56:09,017 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1 2023-11-27 14:56:09,017 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f53154fc29f582902661898287c317b5 columnFamilyName family1 2023-11-27 14:56:09,026 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-19921ab818304b7ca96849e3388bd514 2023-11-27 14:56:09,031 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-44d0a27b9dfe46dcb2a46922bc5cb794 2023-11-27 14:56:09,036 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-73d0bbf7e9cb43e3a1561e0e7e56f260 2023-11-27 14:56:09,040 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-8bb35dffb04542408445e0a661ed6843 2023-11-27 14:56:09,041 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(310): Store=f53154fc29f582902661898287c317b5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:09,041 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f53154fc29f582902661898287c317b5 2023-11-27 14:56:09,042 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2 2023-11-27 14:56:09,042 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2 2023-11-27 14:56:09,042 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f53154fc29f582902661898287c317b5 columnFamilyName family2 2023-11-27 14:56:09,048 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-06188b30d7f149fc989a47c75248d5eb 2023-11-27 14:56:09,052 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-384e3e9c79f642ecaf650fef248f348e 2023-11-27 14:56:09,057 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-be1d45f3a67c4f3eaeea2cd5482139f0 2023-11-27 14:56:09,062 DEBUG [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-f646c62f8dfd4699bf74c0db3d8ca297 2023-11-27 14:56:09,062 INFO [StoreOpener-f53154fc29f582902661898287c317b5-1] regionserver.HStore(310): Store=f53154fc29f582902661898287c317b5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:09,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5 2023-11-27 14:56:09,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5 2023-11-27 14:56:09,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:56:09,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f53154fc29f582902661898287c317b5 2023-11-27 14:56:09,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f53154fc29f582902661898287c317b5; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70532865, jitterRate=0.05102159082889557}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:56:09,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:56:09,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5., pid=103, masterSystemTime=1701096969011 2023-11-27 14:56:09,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:56:09,069 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-27 14:56:09,069 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-27 14:56:09,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-27 14:56:09,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:09,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:09,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:09,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d2db448818547f8516ae9f9e43b817b6, NAME => 'testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.', STARTKEY => '2', ENDKEY => ''} 2023-11-27 14:56:09,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:09,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:09,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:09,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:09,073 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=f53154fc29f582902661898287c317b5, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:09,073 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701096969073"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096969073"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096969073"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096969073"}]},"ts":"1701096969073"} 2023-11-27 14:56:09,074 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-27 14:56:09,074 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1912): f53154fc29f582902661898287c317b5/family1 is initiating minor compaction (all files) 2023-11-27 14:56:09,074 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f53154fc29f582902661898287c317b5/family1 in testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:09,074 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-73d0bbf7e9cb43e3a1561e0e7e56f260, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-44d0a27b9dfe46dcb2a46922bc5cb794, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-8bb35dffb04542408445e0a661ed6843, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-19921ab818304b7ca96849e3388bd514] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp, totalSize=21.6 K 2023-11-27 14:56:09,075 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-73d0bbf7e9cb43e3a1561e0e7e56f260, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946676 2023-11-27 14:56:09,076 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-44d0a27b9dfe46dcb2a46922bc5cb794, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:56:09,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=103, resume processing ppid=99 2023-11-27 14:56:09,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure f53154fc29f582902661898287c317b5, server=jenkins-hbase4.apache.org,42669,1701096850360 in 214 msec 2023-11-27 14:56:09,077 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-8bb35dffb04542408445e0a661ed6843, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950264 2023-11-27 14:56:09,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=f53154fc29f582902661898287c317b5, ASSIGN in 373 msec 2023-11-27 14:56:09,083 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-19921ab818304b7ca96849e3388bd514, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952063 2023-11-27 14:56:09,085 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-27 14:56:09,085 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HStore(1912): f53154fc29f582902661898287c317b5/family2 is initiating minor compaction (all files) 2023-11-27 14:56:09,085 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HRegion(2259): Starting compaction of f53154fc29f582902661898287c317b5/family2 in testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:09,085 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-06188b30d7f149fc989a47c75248d5eb, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-f646c62f8dfd4699bf74c0db3d8ca297, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-be1d45f3a67c4f3eaeea2cd5482139f0, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-384e3e9c79f642ecaf650fef248f348e] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp, totalSize=20.4 K 2023-11-27 14:56:09,086 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-06188b30d7f149fc989a47c75248d5eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946676 2023-11-27 14:56:09,087 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-f646c62f8dfd4699bf74c0db3d8ca297, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:56:09,087 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-be1d45f3a67c4f3eaeea2cd5482139f0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950264 2023-11-27 14:56:09,088 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-384e3e9c79f642ecaf650fef248f348e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952063 2023-11-27 14:56:09,098 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:09,101 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1 2023-11-27 14:56:09,103 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1 2023-11-27 14:56:09,104 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2db448818547f8516ae9f9e43b817b6 columnFamilyName family1 2023-11-27 14:56:09,119 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-54bff7e3315248878a316aabec8f5fb8 2023-11-27 14:56:09,139 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-83ab5ebd1f744748a511df6081b4ca4e 2023-11-27 14:56:09,157 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-9c5c6b42e7aa4057b2d2525439abb851 2023-11-27 14:56:09,159 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] throttle.PressureAwareThroughputController(145): f53154fc29f582902661898287c317b5#family2#compaction#112 average throughput is 0.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:56:09,159 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): f53154fc29f582902661898287c317b5#family1#compaction#113 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:56:09,174 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-e77c71cc85014290a851fe41a0dc07ba 2023-11-27 14:56:09,174 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(310): Store=d2db448818547f8516ae9f9e43b817b6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:09,175 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:09,176 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2 2023-11-27 14:56:09,176 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2 2023-11-27 14:56:09,176 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2db448818547f8516ae9f9e43b817b6 columnFamilyName family2 2023-11-27 14:56:09,187 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-383ab85ed5fa46a19088c45d22cb1c6f 2023-11-27 14:56:09,192 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-55af4ee02b754f3e80ab86a4399ba221 2023-11-27 14:56:09,196 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-7cade81205724088b52161b761cad33c 2023-11-27 14:56:09,200 DEBUG [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-aacaf93d059f41879ce21e42a7deb376 2023-11-27 14:56:09,200 INFO [StoreOpener-d2db448818547f8516ae9f9e43b817b6-1] regionserver.HStore(310): Store=d2db448818547f8516ae9f9e43b817b6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:09,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:09,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:09,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:56:09,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:09,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d2db448818547f8516ae9f9e43b817b6; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74987225, jitterRate=0.11739672720432281}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:56:09,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:56:09,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6., pid=101, masterSystemTime=1701096969011 2023-11-27 14:56:09,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-27 14:56:09,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-27 14:56:09,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:09,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:09,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:09,207 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=d2db448818547f8516ae9f9e43b817b6, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:09,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 68bb86a7752ed0247b952754e90231e7, NAME => 'testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.', STARTKEY => '', ENDKEY => '1'} 2023-11-27 14:56:09,207 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096969207"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096969207"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096969207"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096969207"}]},"ts":"1701096969207"} 2023-11-27 14:56:09,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:09,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:09,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:09,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:09,209 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:09,210 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1 2023-11-27 14:56:09,210 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1 2023-11-27 14:56:09,210 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=101, resume processing ppid=100 2023-11-27 14:56:09,210 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure d2db448818547f8516ae9f9e43b817b6, server=jenkins-hbase4.apache.org,42669,1701096850360 in 348 msec 2023-11-27 14:56:09,210 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 68bb86a7752ed0247b952754e90231e7 columnFamilyName family1 2023-11-27 14:56:09,212 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d2db448818547f8516ae9f9e43b817b6, ASSIGN in 507 msec 2023-11-27 14:56:09,223 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-11dbd4e159484772a2f6b1ba3218b84f 2023-11-27 14:56:09,228 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-6cae8ebbf33443cea0a04ac326c2c78e 2023-11-27 14:56:09,235 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f3476c76367a4094959a000e438862a7 2023-11-27 14:56:09,241 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f90fc4bfa7f84ce18764cfcbd9c5c38e 2023-11-27 14:56:09,241 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(310): Store=68bb86a7752ed0247b952754e90231e7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:09,241 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:09,242 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2 2023-11-27 14:56:09,242 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2 2023-11-27 14:56:09,243 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 68bb86a7752ed0247b952754e90231e7 columnFamilyName family2 2023-11-27 14:56:09,256 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-3f82744f5963467498262e333eab0bca 2023-11-27 14:56:09,262 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-62cd5b0e36874070bd39a34ab752b1a6 2023-11-27 14:56:09,285 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-7362e669758d4b249076960802af5b98 2023-11-27 14:56:09,293 DEBUG [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(539): loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-ac972ceb881e40c88492f6d98ccfde53 2023-11-27 14:56:09,293 INFO [StoreOpener-68bb86a7752ed0247b952754e90231e7-1] regionserver.HStore(310): Store=68bb86a7752ed0247b952754e90231e7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:09,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:09,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:09,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:56:09,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:09,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 68bb86a7752ed0247b952754e90231e7; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59473144, jitterRate=-0.11378109455108643}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:56:09,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:56:09,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7., pid=102, masterSystemTime=1701096969011 2023-11-27 14:56:09,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-27 14:56:09,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-27 14:56:09,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:09,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:09,301 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=68bb86a7752ed0247b952754e90231e7, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:09,301 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1701096969301"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096969301"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096969301"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096969301"}]},"ts":"1701096969301"} 2023-11-27 14:56:09,304 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=102, resume processing ppid=98 2023-11-27 14:56:09,304 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure 68bb86a7752ed0247b952754e90231e7, server=jenkins-hbase4.apache.org,42669,1701096850360 in 442 msec 2023-11-27 14:56:09,305 INFO [PEWorker-4] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=98, resume processing ppid=97 2023-11-27 14:56:09,305 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=68bb86a7752ed0247b952754e90231e7, ASSIGN in 601 msec 2023-11-27 14:56:09,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096969306"}]},"ts":"1701096969306"} 2023-11-27 14:56:09,307 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-11-27 14:56:09,309 INFO [PEWorker-1] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-11-27 14:56:09,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 620 msec 2023-11-27 14:56:09,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-27 14:56:09,317 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-11-27 14:56:09,336 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:09,351 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:09,362 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:09,363 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:09,372 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-27 14:56:09,395 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-11-27 14:56:09,395 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:09,397 DEBUG [Listener at localhost/37439] zookeeper.ReadOnlyZKClient(139): Connect 0x1f518260 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:56:09,416 DEBUG [Listener at localhost/37439] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60be19c4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:56:09,417 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:09,417 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:09,417 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:09,966 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311272f04ad245a1a4450a4b5786a0ce3becb to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311272f04ad245a1a4450a4b5786a0ce3becb 2023-11-27 14:56:09,967 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:09,980 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:56:09,984 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43714, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:56:09,990 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family2/15d52e00965d498893fa1f90e1f34a13 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/15d52e00965d498893fa1f90e1f34a13 2023-11-27 14:56:09,991 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/.tmp/family1/c508b6fc4d2f401e9c2e7b3d0875244e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/c508b6fc4d2f401e9c2e7b3d0875244e 2023-11-27 14:56:09,992 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:56:09,997 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38418, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:56:09,997 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in f53154fc29f582902661898287c317b5/family2 of f53154fc29f582902661898287c317b5 into 15d52e00965d498893fa1f90e1f34a13(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:09,997 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:56:09,997 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5., storeName=f53154fc29f582902661898287c317b5/family2, priority=12, startTime=1701096969069; duration=0sec 2023-11-27 14:56:09,997 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in f53154fc29f582902661898287c317b5/family1 of f53154fc29f582902661898287c317b5 into c508b6fc4d2f401e9c2e7b3d0875244e(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:09,997 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-27 14:56:09,997 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:56:09,998 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5., storeName=f53154fc29f582902661898287c317b5/family1, priority=12, startTime=1701096969069; duration=0sec 2023-11-27 14:56:09,998 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-27 14:56:09,998 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-27 14:56:09,998 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-27 14:56:10,007 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-27 14:56:10,008 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HStore(1912): d2db448818547f8516ae9f9e43b817b6/family1 is initiating minor compaction (all files) 2023-11-27 14:56:10,008 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HRegion(2259): Starting compaction of d2db448818547f8516ae9f9e43b817b6/family1 in testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:10,008 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-9c5c6b42e7aa4057b2d2525439abb851, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-54bff7e3315248878a316aabec8f5fb8, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-83ab5ebd1f744748a511df6081b4ca4e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-e77c71cc85014290a851fe41a0dc07ba] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp, totalSize=21.6 K 2023-11-27 14:56:10,008 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-27 14:56:10,008 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1912): 68bb86a7752ed0247b952754e90231e7/family2 is initiating minor compaction (all files) 2023-11-27 14:56:10,008 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 68bb86a7752ed0247b952754e90231e7/family2 in testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:10,008 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-ac972ceb881e40c88492f6d98ccfde53, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-62cd5b0e36874070bd39a34ab752b1a6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-3f82744f5963467498262e333eab0bca, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-7362e669758d4b249076960802af5b98] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp, totalSize=20.4 K 2023-11-27 14:56:10,012 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-ac972ceb881e40c88492f6d98ccfde53, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946675 2023-11-27 14:56:10,013 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-9c5c6b42e7aa4057b2d2525439abb851, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946677 2023-11-27 14:56:10,013 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-62cd5b0e36874070bd39a34ab752b1a6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:56:10,016 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-54bff7e3315248878a316aabec8f5fb8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:56:10,016 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-3f82744f5963467498262e333eab0bca, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950263 2023-11-27 14:56:10,016 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-83ab5ebd1f744748a511df6081b4ca4e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950264 2023-11-27 14:56:10,017 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-7362e669758d4b249076960802af5b98, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952063 2023-11-27 14:56:10,018 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127/family1/ed6b9bbb7e9a4369951bc54632de5b6d first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-11-27 14:56:10,019 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-e77c71cc85014290a851fe41a0dc07ba, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952064 2023-11-27 14:56:10,019 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127/family1/ed6b9bbb7e9a4369951bc54632de5b6d}] 2023-11-27 14:56:10,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127/family1/ed6b9bbb7e9a4369951bc54632de5b6d for inclusion in f53154fc29f582902661898287c317b5/family1 2023-11-27 14:56:10,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-11-27 14:56:10,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-27 14:56:10,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:56:10,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127/family1/ed6b9bbb7e9a4369951bc54632de5b6d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__poklvu6tmnjnrpra8stleqh4vibf6u8lp3ul7okvsl3qb9di2oirgscunebqstht/family1/ed6b9bbb7e9a4369951bc54632de5b6d 2023-11-27 14:56:10,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__poklvu6tmnjnrpra8stleqh4vibf6u8lp3ul7okvsl3qb9di2oirgscunebqstht/family1/ed6b9bbb7e9a4369951bc54632de5b6d as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/11af74a210e94b3c842e2178039542f1_SeqId_36_ 2023-11-27 14:56:10,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__poklvu6tmnjnrpra8stleqh4vibf6u8lp3ul7okvsl3qb9di2oirgscunebqstht/family1/ed6b9bbb7e9a4369951bc54632de5b6d into f53154fc29f582902661898287c317b5/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/11af74a210e94b3c842e2178039542f1_SeqId_36_ - updating store file list. 2023-11-27 14:56:10,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/11af74a210e94b3c842e2178039542f1_SeqId_36_ into f53154fc29f582902661898287c317b5/family1 2023-11-27 14:56:10,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__poklvu6tmnjnrpra8stleqh4vibf6u8lp3ul7okvsl3qb9di2oirgscunebqstht/family1/ed6b9bbb7e9a4369951bc54632de5b6d into f53154fc29f582902661898287c317b5/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/11af74a210e94b3c842e2178039542f1_SeqId_36_) 2023-11-27 14:56:10,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__poklvu6tmnjnrpra8stleqh4vibf6u8lp3ul7okvsl3qb9di2oirgscunebqstht/family1/ed6b9bbb7e9a4369951bc54632de5b6d 2023-11-27 14:56:10,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-73d0bbf7e9cb43e3a1561e0e7e56f260, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-44d0a27b9dfe46dcb2a46922bc5cb794, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-8bb35dffb04542408445e0a661ed6843, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-19921ab818304b7ca96849e3388bd514] to archive 2023-11-27 14:56:10,050 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:10,050 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:10,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:10,052 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-73d0bbf7e9cb43e3a1561e0e7e56f260 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-73d0bbf7e9cb43e3a1561e0e7e56f260 2023-11-27 14:56:10,053 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311276c172b1bef3842fea773178819944ccb to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311276c172b1bef3842fea773178819944ccb 2023-11-27 14:56:10,054 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-44d0a27b9dfe46dcb2a46922bc5cb794 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-44d0a27b9dfe46dcb2a46922bc5cb794 2023-11-27 14:56:10,055 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231127ae5335fb8ba24277a60870ae7f9a4267 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231127ae5335fb8ba24277a60870ae7f9a4267 2023-11-27 14:56:10,056 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-8bb35dffb04542408445e0a661ed6843 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-8bb35dffb04542408445e0a661ed6843 2023-11-27 14:56:10,056 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231127ecf30d17888a4c878b4e64030723965f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231127ecf30d17888a4c878b4e64030723965f 2023-11-27 14:56:10,057 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-19921ab818304b7ca96849e3388bd514 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family1/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-19921ab818304b7ca96849e3388bd514 2023-11-27 14:56:10,059 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-06188b30d7f149fc989a47c75248d5eb, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-f646c62f8dfd4699bf74c0db3d8ca297, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-be1d45f3a67c4f3eaeea2cd5482139f0, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-384e3e9c79f642ecaf650fef248f348e] to archive 2023-11-27 14:56:10,060 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 68bb86a7752ed0247b952754e90231e7#family2#compaction#114 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:56:10,064 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:10,064 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231127f4b90f9713bc44fc9fb9c60ce591c905 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231127f4b90f9713bc44fc9fb9c60ce591c905 2023-11-27 14:56:10,064 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-27 14:56:10,064 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231127' 2023-11-27 14:56:10,066 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:10,068 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-06188b30d7f149fc989a47c75248d5eb to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-06188b30d7f149fc989a47c75248d5eb 2023-11-27 14:56:10,069 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-f646c62f8dfd4699bf74c0db3d8ca297 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-f646c62f8dfd4699bf74c0db3d8ca297 2023-11-27 14:56:10,070 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-be1d45f3a67c4f3eaeea2cd5482139f0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-be1d45f3a67c4f3eaeea2cd5482139f0 2023-11-27 14:56:10,072 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-384e3e9c79f642ecaf650fef248f348e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/family2/testCompactionWithHFileLink=f53154fc29f582902661898287c317b5-384e3e9c79f642ecaf650fef248f348e 2023-11-27 14:56:10,073 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] throttle.PressureAwareThroughputController(145): d2db448818547f8516ae9f9e43b817b6#family1#compaction#115 average throughput is 0.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:56:10,345 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112792f9e670061e400c856f0759d49b4fb2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023112792f9e670061e400c856f0759d49b4fb2 2023-11-27 14:56:10,346 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:10,357 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127a3ae6fdb13234747bf8ade4741b0f354 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231127a3ae6fdb13234747bf8ade4741b0f354 2023-11-27 14:56:10,357 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:10,383 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127/family1/07e54ebfd3a649e7baefb2f09bc1fb1d first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-11-27 14:56:10,383 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127/family1/07e54ebfd3a649e7baefb2f09bc1fb1d}] 2023-11-27 14:56:10,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127/family1/07e54ebfd3a649e7baefb2f09bc1fb1d for inclusion in d2db448818547f8516ae9f9e43b817b6/family1 2023-11-27 14:56:10,391 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127/family1/a758a93c764746eca3686c51f198666b first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-11-27 14:56:10,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-11-27 14:56:10,392 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=32 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127/family1/a758a93c764746eca3686c51f198666b}] 2023-11-27 14:56:10,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-27 14:56:10,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:56:10,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127/family1/07e54ebfd3a649e7baefb2f09bc1fb1d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__fkfunrmiuprs3fhso3gb4vg770tfs1trp96mfi3b0n7u55cjgoa17tnbur67m61/family1/07e54ebfd3a649e7baefb2f09bc1fb1d 2023-11-27 14:56:10,395 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127/family1/a758a93c764746eca3686c51f198666b for inclusion in 68bb86a7752ed0247b952754e90231e7/family1 2023-11-27 14:56:10,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__fkfunrmiuprs3fhso3gb4vg770tfs1trp96mfi3b0n7u55cjgoa17tnbur67m61/family1/07e54ebfd3a649e7baefb2f09bc1fb1d as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/d05ace67731f466f80c7765b250f577b_SeqId_34_ 2023-11-27 14:56:10,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__fkfunrmiuprs3fhso3gb4vg770tfs1trp96mfi3b0n7u55cjgoa17tnbur67m61/family1/07e54ebfd3a649e7baefb2f09bc1fb1d into d2db448818547f8516ae9f9e43b817b6/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/d05ace67731f466f80c7765b250f577b_SeqId_34_ - updating store file list. 2023-11-27 14:56:10,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-11-27 14:56:10,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last=1 2023-11-27 14:56:10,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:56:10,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127/family1/a758a93c764746eca3686c51f198666b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__4r7jsgqc2ia1r5voiia458201ackrniam7a211an9mkaj4vkmq7j7dd13ck6o0ji/family1/a758a93c764746eca3686c51f198666b 2023-11-27 14:56:10,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/d05ace67731f466f80c7765b250f577b_SeqId_34_ into d2db448818547f8516ae9f9e43b817b6/family1 2023-11-27 14:56:10,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__fkfunrmiuprs3fhso3gb4vg770tfs1trp96mfi3b0n7u55cjgoa17tnbur67m61/family1/07e54ebfd3a649e7baefb2f09bc1fb1d into d2db448818547f8516ae9f9e43b817b6/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/d05ace67731f466f80c7765b250f577b_SeqId_34_) 2023-11-27 14:56:10,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__4r7jsgqc2ia1r5voiia458201ackrniam7a211an9mkaj4vkmq7j7dd13ck6o0ji/family1/a758a93c764746eca3686c51f198666b as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/64f06979c2cc45c995e64a409125d039_SeqId_34_ 2023-11-27 14:56:10,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__fkfunrmiuprs3fhso3gb4vg770tfs1trp96mfi3b0n7u55cjgoa17tnbur67m61/family1/07e54ebfd3a649e7baefb2f09bc1fb1d 2023-11-27 14:56:10,401 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__4r7jsgqc2ia1r5voiia458201ackrniam7a211an9mkaj4vkmq7j7dd13ck6o0ji/family1/a758a93c764746eca3686c51f198666b into 68bb86a7752ed0247b952754e90231e7/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/64f06979c2cc45c995e64a409125d039_SeqId_34_ - updating store file list. 2023-11-27 14:56:10,403 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:10,404 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:10,405 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/64f06979c2cc45c995e64a409125d039_SeqId_34_ into 68bb86a7752ed0247b952754e90231e7/family1 2023-11-27 14:56:10,405 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__4r7jsgqc2ia1r5voiia458201ackrniam7a211an9mkaj4vkmq7j7dd13ck6o0ji/family1/a758a93c764746eca3686c51f198666b into 68bb86a7752ed0247b952754e90231e7/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/64f06979c2cc45c995e64a409125d039_SeqId_34_) 2023-11-27 14:56:10,406 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311273f6fd9351e9842df8dddeb3f1bd00e91 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311273f6fd9351e9842df8dddeb3f1bd00e91 2023-11-27 14:56:10,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testCompactionWithHFileLink__4r7jsgqc2ia1r5voiia458201ackrniam7a211an9mkaj4vkmq7j7dd13ck6o0ji/family1/a758a93c764746eca3686c51f198666b 2023-11-27 14:56:10,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:10,407 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311274a4801ad97a94826a437fba4aaced078 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311274a4801ad97a94826a437fba4aaced078 2023-11-27 14:56:10,407 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311279f6d46bf55ff44f8a38c99407a29d75f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311279f6d46bf55ff44f8a38c99407a29d75f 2023-11-27 14:56:10,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-11-27 14:56:10,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(1912): 68bb86a7752ed0247b952754e90231e7/family1 is initiating minor compaction (all files) 2023-11-27 14:56:10,408 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231127fedadc64d2504dccb03740338659f0c6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231127fedadc64d2504dccb03740338659f0c6 2023-11-27 14:56:10,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@24a6b8c5; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-27 14:56:10,408 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-27 14:56:10,409 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231127' 2023-11-27 14:56:10,409 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:10,410 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:10,411 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:10,412 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023112743713d96bf2d4fa7878b42c12e1a7fe5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023112743713d96bf2d4fa7878b42c12e1a7fe5 2023-11-27 14:56:10,413 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311278bd748d50ec449b591ab4997d2b6bb2a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311278bd748d50ec449b591ab4997d2b6bb2a 2023-11-27 14:56:10,414 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231127ca141289f66b4b37a06eaec5c63297ae to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231127ca141289f66b4b37a06eaec5c63297ae 2023-11-27 14:56:10,415 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231127deae02fa52974d90841c7ec64ee944df to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231127deae02fa52974d90841c7ec64ee944df 2023-11-27 14:56:10,415 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-27 14:56:10,415 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231127' 2023-11-27 14:56:10,416 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231127 for table='testCompactionWithHFileLink' and column='family1' 2023-11-27 14:56:10,416 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-11-27 14:56:10,418 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:10,422 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:10,425 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:10,426 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:10,432 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:10,433 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:10,456 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=289 (was 239) - Thread LEAK? -, OpenFileDescriptor=790 (was 630) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=182 (was 225), ProcessCount=168 (was 168), AvailableMemoryMB=8434 (was 8124) - AvailableMemoryMB LEAK? - 2023-11-27 14:56:10,473 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=289, OpenFileDescriptor=790, MaxFileDescriptor=60000, SystemLoadAverage=182, ProcessCount=168, AvailableMemoryMB=8433 2023-11-27 14:56:10,480 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-11-27 14:56:10,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-11-27 14:56:10,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-11-27 14:56:10,490 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-27 14:56:10,494 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 13 msec 2023-11-27 14:56:10,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-11-27 14:56:10,590 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-27 14:56:10,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-11-27 14:56:10,592 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:56:10,592 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-11-27 14:56:10,593 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:56:10,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-27 14:56:10,595 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972 2023-11-27 14:56:10,595 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:10,595 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:10,595 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972 empty. 2023-11-27 14:56:10,595 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7 empty. 2023-11-27 14:56:10,596 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867 empty. 2023-11-27 14:56:10,596 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972 2023-11-27 14:56:10,596 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:10,596 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:10,596 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-11-27 14:56:10,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-27 14:56:10,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-27 14:56:10,916 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family2/0204bc79e6a540e383449679fbd91e59 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/0204bc79e6a540e383449679fbd91e59 2023-11-27 14:56:10,920 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 68bb86a7752ed0247b952754e90231e7/family2 of 68bb86a7752ed0247b952754e90231e7 into 0204bc79e6a540e383449679fbd91e59(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:10,920 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:56:10,920 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7., storeName=68bb86a7752ed0247b952754e90231e7/family2, priority=12, startTime=1701096969299; duration=0sec 2023-11-27 14:56:10,920 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-27 14:56:10,921 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 68bb86a7752ed0247b952754e90231e7/family1 in testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:10,921 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-11dbd4e159484772a2f6b1ba3218b84f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f3476c76367a4094959a000e438862a7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-6cae8ebbf33443cea0a04ac326c2c78e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f90fc4bfa7f84ce18764cfcbd9c5c38e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/64f06979c2cc45c995e64a409125d039_SeqId_34_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp, totalSize=28.9 K 2023-11-27 14:56:10,921 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-11dbd4e159484772a2f6b1ba3218b84f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946675 2023-11-27 14:56:10,922 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f3476c76367a4094959a000e438862a7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:56:10,922 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-6cae8ebbf33443cea0a04ac326c2c78e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950263 2023-11-27 14:56:10,923 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f90fc4bfa7f84ce18764cfcbd9c5c38e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952063 2023-11-27 14:56:10,923 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 64f06979c2cc45c995e64a409125d039_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1701096946675 2023-11-27 14:56:10,942 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): 68bb86a7752ed0247b952754e90231e7#family1#compaction#116 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:56:10,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-ac972ceb881e40c88492f6d98ccfde53, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-62cd5b0e36874070bd39a34ab752b1a6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-3f82744f5963467498262e333eab0bca, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-7362e669758d4b249076960802af5b98] to archive 2023-11-27 14:56:10,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:10,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-ac972ceb881e40c88492f6d98ccfde53 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-ac972ceb881e40c88492f6d98ccfde53 2023-11-27 14:56:10,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-62cd5b0e36874070bd39a34ab752b1a6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-62cd5b0e36874070bd39a34ab752b1a6 2023-11-27 14:56:10,958 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-3f82744f5963467498262e333eab0bca to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-3f82744f5963467498262e333eab0bca 2023-11-27 14:56:10,959 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-7362e669758d4b249076960802af5b98 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family2/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-7362e669758d4b249076960802af5b98 2023-11-27 14:56:11,003 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-11-27 14:56:11,004 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3404a93afc957274c1f21854ed231972, NAME => 'ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:56:11,004 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 1cf135bbee78d983093ed681ad9bc867, NAME => 'ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:56:11,005 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 480bebe747409b1ccbbfa174b76957d7, NAME => 'ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:56:11,018 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:11,018 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 480bebe747409b1ccbbfa174b76957d7, disabling compactions & flushes 2023-11-27 14:56:11,018 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:11,018 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:11,018 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. after waiting 0 ms 2023-11-27 14:56:11,018 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:11,018 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:11,018 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:11,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-27 14:56:11,316 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:56:11,319 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family1/08a9b3fc2bb248fda5fd71988917e492 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/08a9b3fc2bb248fda5fd71988917e492 2023-11-27 14:56:11,324 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d2db448818547f8516ae9f9e43b817b6/family1 of d2db448818547f8516ae9f9e43b817b6 into 08a9b3fc2bb248fda5fd71988917e492(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:11,324 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:56:11,324 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6., storeName=d2db448818547f8516ae9f9e43b817b6/family1, priority=12, startTime=1701096969205; duration=1sec 2023-11-27 14:56:11,325 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-27 14:56:11,325 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 5 compacting, 0 eligible, 16 blocking 2023-11-27 14:56:11,325 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-27 14:56:11,325 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-27 14:56:11,325 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. because compaction request was cancelled 2023-11-27 14:56:11,325 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-27 14:56:11,327 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-27 14:56:11,327 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HStore(1912): d2db448818547f8516ae9f9e43b817b6/family2 is initiating minor compaction (all files) 2023-11-27 14:56:11,327 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HRegion(2259): Starting compaction of d2db448818547f8516ae9f9e43b817b6/family2 in testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:11,327 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-aacaf93d059f41879ce21e42a7deb376, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-55af4ee02b754f3e80ab86a4399ba221, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-7cade81205724088b52161b761cad33c, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-383ab85ed5fa46a19088c45d22cb1c6f] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp, totalSize=20.4 K 2023-11-27 14:56:11,328 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-aacaf93d059f41879ce21e42a7deb376, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096946677 2023-11-27 14:56:11,329 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-55af4ee02b754f3e80ab86a4399ba221, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096948478 2023-11-27 14:56:11,329 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-7cade81205724088b52161b761cad33c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096950264 2023-11-27 14:56:11,330 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-383ab85ed5fa46a19088c45d22cb1c6f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096952064 2023-11-27 14:56:11,350 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-9c5c6b42e7aa4057b2d2525439abb851, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-54bff7e3315248878a316aabec8f5fb8, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-83ab5ebd1f744748a511df6081b4ca4e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-e77c71cc85014290a851fe41a0dc07ba] to archive 2023-11-27 14:56:11,351 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:11,352 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] throttle.PressureAwareThroughputController(145): d2db448818547f8516ae9f9e43b817b6#family2#compaction#117 average throughput is 0.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-27 14:56:11,353 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-9c5c6b42e7aa4057b2d2525439abb851 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-9c5c6b42e7aa4057b2d2525439abb851 2023-11-27 14:56:11,355 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-54bff7e3315248878a316aabec8f5fb8 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-54bff7e3315248878a316aabec8f5fb8 2023-11-27 14:56:11,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-83ab5ebd1f744748a511df6081b4ca4e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-83ab5ebd1f744748a511df6081b4ca4e 2023-11-27 14:56:11,358 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-e77c71cc85014290a851fe41a0dc07ba to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family1/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-e77c71cc85014290a851fe41a0dc07ba 2023-11-27 14:56:11,374 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/.tmp/family2/8056ae97c8da4873a66d0746474dca59 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/8056ae97c8da4873a66d0746474dca59 2023-11-27 14:56:11,379 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d2db448818547f8516ae9f9e43b817b6/family2 of d2db448818547f8516ae9f9e43b817b6 into 8056ae97c8da4873a66d0746474dca59(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:11,379 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:56:11,379 INFO [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6., storeName=d2db448818547f8516ae9f9e43b817b6/family2, priority=12, startTime=1701096969205; duration=0sec 2023-11-27 14:56:11,380 DEBUG [RS:0;jenkins-hbase4:42669-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 1cf135bbee78d983093ed681ad9bc867, disabling compactions & flushes 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 3404a93afc957274c1f21854ed231972, disabling compactions & flushes 2023-11-27 14:56:11,419 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:11,419 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. after waiting 0 ms 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. after waiting 0 ms 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:11,419 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:11,419 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:11,419 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:11,421 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:56:11,422 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701096971422"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096971422"}]},"ts":"1701096971422"} 2023-11-27 14:56:11,423 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1701096971422"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096971422"}]},"ts":"1701096971422"} 2023-11-27 14:56:11,423 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701096971422"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096971422"}]},"ts":"1701096971422"} 2023-11-27 14:56:11,425 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-27 14:56:11,425 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:56:11,425 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096971425"}]},"ts":"1701096971425"} 2023-11-27 14:56:11,427 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-11-27 14:56:11,434 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3404a93afc957274c1f21854ed231972, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=1cf135bbee78d983093ed681ad9bc867, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=480bebe747409b1ccbbfa174b76957d7, ASSIGN}] 2023-11-27 14:56:11,436 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=480bebe747409b1ccbbfa174b76957d7, ASSIGN 2023-11-27 14:56:11,436 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3404a93afc957274c1f21854ed231972, ASSIGN 2023-11-27 14:56:11,436 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=1cf135bbee78d983093ed681ad9bc867, ASSIGN 2023-11-27 14:56:11,437 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=1cf135bbee78d983093ed681ad9bc867, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:56:11,437 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=480bebe747409b1ccbbfa174b76957d7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:56:11,437 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3404a93afc957274c1f21854ed231972, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:56:11,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-aacaf93d059f41879ce21e42a7deb376, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-55af4ee02b754f3e80ab86a4399ba221, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-7cade81205724088b52161b761cad33c, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-383ab85ed5fa46a19088c45d22cb1c6f] to archive 2023-11-27 14:56:11,457 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:11,459 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-aacaf93d059f41879ce21e42a7deb376 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-aacaf93d059f41879ce21e42a7deb376 2023-11-27 14:56:11,461 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-55af4ee02b754f3e80ab86a4399ba221 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-55af4ee02b754f3e80ab86a4399ba221 2023-11-27 14:56:11,462 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-7cade81205724088b52161b761cad33c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-7cade81205724088b52161b761cad33c 2023-11-27 14:56:11,463 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-383ab85ed5fa46a19088c45d22cb1c6f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/family2/testCompactionWithHFileLink=d2db448818547f8516ae9f9e43b817b6-383ab85ed5fa46a19088c45d22cb1c6f 2023-11-27 14:56:11,590 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=1cf135bbee78d983093ed681ad9bc867, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:11,590 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=480bebe747409b1ccbbfa174b76957d7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:11,590 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=3404a93afc957274c1f21854ed231972, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:11,590 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1701096971590"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096971590"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096971590"}]},"ts":"1701096971590"} 2023-11-27 14:56:11,590 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701096971590"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096971590"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096971590"}]},"ts":"1701096971590"} 2023-11-27 14:56:11,590 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701096971590"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096971590"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096971590"}]},"ts":"1701096971590"} 2023-11-27 14:56:11,592 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure 1cf135bbee78d983093ed681ad9bc867, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:11,593 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure 3404a93afc957274c1f21854ed231972, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:11,593 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure 480bebe747409b1ccbbfa174b76957d7, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:11,645 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.15384615384615385, tune throughput to 57.69 MB/second 2023-11-27 14:56:11,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-27 14:56:11,743 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:11,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:11,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1cf135bbee78d983093ed681ad9bc867, NAME => 'ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867.', STARTKEY => '1', ENDKEY => '2'} 2023-11-27 14:56:11,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:11,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:11,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:11,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:11,749 INFO [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:11,751 DEBUG [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1 2023-11-27 14:56:11,751 DEBUG [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1 2023-11-27 14:56:11,751 INFO [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1cf135bbee78d983093ed681ad9bc867 columnFamilyName family1 2023-11-27 14:56:11,753 INFO [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] regionserver.HStore(310): Store=1cf135bbee78d983093ed681ad9bc867/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:11,753 INFO [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:11,754 DEBUG [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2 2023-11-27 14:56:11,754 DEBUG [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2 2023-11-27 14:56:11,754 INFO [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1cf135bbee78d983093ed681ad9bc867 columnFamilyName family2 2023-11-27 14:56:11,756 INFO [StoreOpener-1cf135bbee78d983093ed681ad9bc867-1] regionserver.HStore(310): Store=1cf135bbee78d983093ed681ad9bc867/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:11,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:11,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:11,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:56:11,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:11,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:56:11,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1cf135bbee78d983093ed681ad9bc867; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61322993, jitterRate=-0.08621619641780853}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:56:11,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:11,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867., pid=109, masterSystemTime=1701096971743 2023-11-27 14:56:11,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:11,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:11,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:11,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 480bebe747409b1ccbbfa174b76957d7, NAME => 'ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7.', STARTKEY => '2', ENDKEY => ''} 2023-11-27 14:56:11,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:11,764 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=1cf135bbee78d983093ed681ad9bc867, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:11,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:11,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:11,764 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1701096971764"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096971764"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096971764"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096971764"}]},"ts":"1701096971764"} 2023-11-27 14:56:11,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:11,765 INFO [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:11,766 DEBUG [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1 2023-11-27 14:56:11,766 DEBUG [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1 2023-11-27 14:56:11,767 INFO [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 480bebe747409b1ccbbfa174b76957d7 columnFamilyName family1 2023-11-27 14:56:11,767 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=109, resume processing ppid=107 2023-11-27 14:56:11,767 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure 1cf135bbee78d983093ed681ad9bc867, server=jenkins-hbase4.apache.org,42669,1701096850360 in 173 msec 2023-11-27 14:56:11,767 INFO [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] regionserver.HStore(310): Store=480bebe747409b1ccbbfa174b76957d7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:11,768 INFO [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:11,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=1cf135bbee78d983093ed681ad9bc867, ASSIGN in 333 msec 2023-11-27 14:56:11,769 DEBUG [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2 2023-11-27 14:56:11,769 DEBUG [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2 2023-11-27 14:56:11,769 INFO [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 480bebe747409b1ccbbfa174b76957d7 columnFamilyName family2 2023-11-27 14:56:11,770 INFO [StoreOpener-480bebe747409b1ccbbfa174b76957d7-1] regionserver.HStore(310): Store=480bebe747409b1ccbbfa174b76957d7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:11,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:11,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:11,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:56:11,779 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/.tmp/family1/6c370e8c83de475a8a2bc2d617503e01 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/6c370e8c83de475a8a2bc2d617503e01 2023-11-27 14:56:11,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:11,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:56:11,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 480bebe747409b1ccbbfa174b76957d7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73712188, jitterRate=0.09839719533920288}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:56:11,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:11,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7., pid=111, masterSystemTime=1701096971743 2023-11-27 14:56:11,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:11,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:11,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:11,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3404a93afc957274c1f21854ed231972, NAME => 'ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972.', STARTKEY => '', ENDKEY => '1'} 2023-11-27 14:56:11,792 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=480bebe747409b1ccbbfa174b76957d7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:11,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 3404a93afc957274c1f21854ed231972 2023-11-27 14:56:11,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:11,793 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701096971792"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096971792"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096971792"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096971792"}]},"ts":"1701096971792"} 2023-11-27 14:56:11,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3404a93afc957274c1f21854ed231972 2023-11-27 14:56:11,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3404a93afc957274c1f21854ed231972 2023-11-27 14:56:11,794 INFO [StoreOpener-3404a93afc957274c1f21854ed231972-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3404a93afc957274c1f21854ed231972 2023-11-27 14:56:11,794 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 68bb86a7752ed0247b952754e90231e7/family1 of 68bb86a7752ed0247b952754e90231e7 into 6c370e8c83de475a8a2bc2d617503e01(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:11,794 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:56:11,794 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1701096970408; duration=0sec 2023-11-27 14:56:11,794 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:56:11,795 DEBUG [StoreOpener-3404a93afc957274c1f21854ed231972-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1 2023-11-27 14:56:11,795 DEBUG [StoreOpener-3404a93afc957274c1f21854ed231972-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1 2023-11-27 14:56:11,796 INFO [StoreOpener-3404a93afc957274c1f21854ed231972-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3404a93afc957274c1f21854ed231972 columnFamilyName family1 2023-11-27 14:56:11,796 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=111, resume processing ppid=108 2023-11-27 14:56:11,797 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure 480bebe747409b1ccbbfa174b76957d7, server=jenkins-hbase4.apache.org,42669,1701096850360 in 201 msec 2023-11-27 14:56:11,797 INFO [StoreOpener-3404a93afc957274c1f21854ed231972-1] regionserver.HStore(310): Store=3404a93afc957274c1f21854ed231972/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:11,797 INFO [StoreOpener-3404a93afc957274c1f21854ed231972-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3404a93afc957274c1f21854ed231972 2023-11-27 14:56:11,798 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=480bebe747409b1ccbbfa174b76957d7, ASSIGN in 363 msec 2023-11-27 14:56:11,798 DEBUG [StoreOpener-3404a93afc957274c1f21854ed231972-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2 2023-11-27 14:56:11,798 DEBUG [StoreOpener-3404a93afc957274c1f21854ed231972-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2 2023-11-27 14:56:11,799 INFO [StoreOpener-3404a93afc957274c1f21854ed231972-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3404a93afc957274c1f21854ed231972 columnFamilyName family2 2023-11-27 14:56:11,799 INFO [StoreOpener-3404a93afc957274c1f21854ed231972-1] regionserver.HStore(310): Store=3404a93afc957274c1f21854ed231972/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:11,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972 2023-11-27 14:56:11,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972 2023-11-27 14:56:11,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:56:11,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3404a93afc957274c1f21854ed231972 2023-11-27 14:56:11,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:56:11,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3404a93afc957274c1f21854ed231972; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62119493, jitterRate=-0.07434742152690887}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:56:11,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:11,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972., pid=110, masterSystemTime=1701096971743 2023-11-27 14:56:11,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:11,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:11,807 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=3404a93afc957274c1f21854ed231972, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:11,808 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1701096971807"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096971807"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096971807"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096971807"}]},"ts":"1701096971807"} 2023-11-27 14:56:11,810 INFO [PEWorker-2] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=110, resume processing ppid=106 2023-11-27 14:56:11,810 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure 3404a93afc957274c1f21854ed231972, server=jenkins-hbase4.apache.org,42669,1701096850360 in 217 msec 2023-11-27 14:56:11,811 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=106, resume processing ppid=105 2023-11-27 14:56:11,811 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3404a93afc957274c1f21854ed231972, ASSIGN in 376 msec 2023-11-27 14:56:11,812 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:56:11,812 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096971812"}]},"ts":"1701096971812"} 2023-11-27 14:56:11,813 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-11-27 14:56:11,815 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:56:11,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2250 sec 2023-11-27 14:56:11,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-11dbd4e159484772a2f6b1ba3218b84f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f3476c76367a4094959a000e438862a7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-6cae8ebbf33443cea0a04ac326c2c78e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f90fc4bfa7f84ce18764cfcbd9c5c38e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/64f06979c2cc45c995e64a409125d039_SeqId_34_] to archive 2023-11-27 14:56:11,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:11,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-11dbd4e159484772a2f6b1ba3218b84f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-11dbd4e159484772a2f6b1ba3218b84f 2023-11-27 14:56:11,857 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f3476c76367a4094959a000e438862a7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f3476c76367a4094959a000e438862a7 2023-11-27 14:56:11,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-6cae8ebbf33443cea0a04ac326c2c78e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-6cae8ebbf33443cea0a04ac326c2c78e 2023-11-27 14:56:11,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f90fc4bfa7f84ce18764cfcbd9c5c38e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/testCompactionWithHFileLink=68bb86a7752ed0247b952754e90231e7-f90fc4bfa7f84ce18764cfcbd9c5c38e 2023-11-27 14:56:11,860 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/64f06979c2cc45c995e64a409125d039_SeqId_34_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/family1/64f06979c2cc45c995e64a409125d039_SeqId_34_ 2023-11-27 14:56:12,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-27 14:56:12,699 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-11-27 14:56:12,704 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:12,705 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:12,705 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:12,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:12,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-27 14:56:12,709 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-27 14:56:12,709 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:12,709 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-27 14:56:12,709 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-27 14:56:12,709 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:12,709 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:12,710 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:12,710 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:12,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:12,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:12,711 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:12,711 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:12,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:12,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:12,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:12,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:12,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-27 14:56:12,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-27 14:56:12,712 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-27 14:56:12,712 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:12,712 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-27 14:56:12,712 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:12,712 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. started... 2023-11-27 14:56:12,714 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:12,714 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:12,714 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:12,714 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. started... 2023-11-27 14:56:12,714 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3404a93afc957274c1f21854ed231972 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-27 14:56:12,714 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:12,714 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 480bebe747409b1ccbbfa174b76957d7 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-27 14:56:12,715 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. started... 2023-11-27 14:56:12,715 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1cf135bbee78d983093ed681ad9bc867 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-27 14:56:13,128 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127f9364d5504ac48c2afce1337e3bc0eb7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231127f9364d5504ac48c2afce1337e3bc0eb7 2023-11-27 14:56:13,128 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311278a8c5e40f9b1419fa3c70bd2ec0d9c6c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311278a8c5e40f9b1419fa3c70bd2ec0d9c6c 2023-11-27 14:56:13,129 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127beca75b363da4fcd8645640e4343a3e5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231127beca75b363da4fcd8645640e4343a3e5 2023-11-27 14:56:13,542 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/59544ee789414c46addbe33bdf3065be 2023-11-27 14:56:13,543 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/2dd63e9a981e4c969641bc1cefa98030 2023-11-27 14:56:13,543 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/f6a1d73e384249a888947261aa58c919 2023-11-27 14:56:13,573 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311272c789c75525f4c158d257e9b9c9bff9a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202311272c789c75525f4c158d257e9b9c9bff9a 2023-11-27 14:56:13,574 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112725fa51d4c72a411cb41ead8a25b51231 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023112725fa51d4c72a411cb41ead8a25b51231 2023-11-27 14:56:13,976 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112701a6aac239a040d6989f713247acda7b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023112701a6aac239a040d6989f713247acda7b 2023-11-27 14:56:13,979 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/fccdfeb0b1a846e393e1178e004dbb50 2023-11-27 14:56:13,980 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/9e15311d2d9b44e29b1d36b369a11fc3 2023-11-27 14:56:13,983 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/2dd63e9a981e4c969641bc1cefa98030 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/2dd63e9a981e4c969641bc1cefa98030 2023-11-27 14:56:13,984 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/f6a1d73e384249a888947261aa58c919 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/f6a1d73e384249a888947261aa58c919 2023-11-27 14:56:13,986 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/2dd63e9a981e4c969641bc1cefa98030, entries=4, sequenceid=5, filesize=5.4 K 2023-11-27 14:56:13,987 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/f6a1d73e384249a888947261aa58c919, entries=4, sequenceid=5, filesize=5.4 K 2023-11-27 14:56:13,988 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/fccdfeb0b1a846e393e1178e004dbb50 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/fccdfeb0b1a846e393e1178e004dbb50 2023-11-27 14:56:13,988 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/9e15311d2d9b44e29b1d36b369a11fc3 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/9e15311d2d9b44e29b1d36b369a11fc3 2023-11-27 14:56:13,991 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/fccdfeb0b1a846e393e1178e004dbb50, entries=2, sequenceid=5, filesize=5.1 K 2023-11-27 14:56:13,992 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/9e15311d2d9b44e29b1d36b369a11fc3, entries=2, sequenceid=5, filesize=5.1 K 2023-11-27 14:56:13,992 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 1cf135bbee78d983093ed681ad9bc867 in 1277ms, sequenceid=5, compaction requested=false 2023-11-27 14:56:13,992 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 3404a93afc957274c1f21854ed231972 in 1278ms, sequenceid=5, compaction requested=false 2023-11-27 14:56:13,992 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-11-27 14:56:13,992 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-11-27 14:56:13,994 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:13,994 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:13,994 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:13,994 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:13,994 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:13,994 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:14,381 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/9878a9d04c334bf993b75c9b88d055e6 2023-11-27 14:56:14,385 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/59544ee789414c46addbe33bdf3065be as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/59544ee789414c46addbe33bdf3065be 2023-11-27 14:56:14,389 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/59544ee789414c46addbe33bdf3065be, entries=4, sequenceid=5, filesize=5.4 K 2023-11-27 14:56:14,389 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/9878a9d04c334bf993b75c9b88d055e6 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/9878a9d04c334bf993b75c9b88d055e6 2023-11-27 14:56:14,393 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/9878a9d04c334bf993b75c9b88d055e6, entries=2, sequenceid=5, filesize=5.1 K 2023-11-27 14:56:14,393 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 480bebe747409b1ccbbfa174b76957d7 in 1679ms, sequenceid=5, compaction requested=false 2023-11-27 14:56:14,394 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:14,394 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-27 14:56:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-27 14:56:14,396 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:14,396 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:14,397 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:14,397 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:14,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-27 14:56:14,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@709d8814[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:14,399 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-27 14:56:14,399 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:14,401 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:14,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:14,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:14,401 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-27 14:56:14,401 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-27 14:56:14,401 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:14,401 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,401 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:14,402 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,402 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-27 14:56:14,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:14,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:14,402 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:14,402 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-27 14:56:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:14,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:14,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:14,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-27 14:56:14,405 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-27 14:56:14,405 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:14,405 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:14,405 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-27 14:56:14,405 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:14,407 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:14,407 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:14,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:14,407 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:14,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:14,407 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:14,409 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:14,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:14,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,412 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,412 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:14,412 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:14,412 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-27 14:56:14,412 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:14,412 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:14,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:14,412 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:14,412 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:14,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:14,412 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-27 14:56:14,412 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:14,413 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:14,413 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:14,413 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:14,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:14,413 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:14,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:14,513 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:14,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:14,515 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:14,515 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:14,515 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:14,518 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:14,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:14,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-27 14:56:14,519 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-27 14:56:14,519 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:14,519 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-27 14:56:14,519 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-27 14:56:14,520 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:14,520 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:14,522 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:14,522 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:14,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:14,523 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:14,523 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:14,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:14,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:14,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:14,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:14,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-27 14:56:14,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-27 14:56:14,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-27 14:56:14,524 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-27 14:56:14,524 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:14,524 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-27 14:56:14,524 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:14,524 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. started... 2023-11-27 14:56:14,524 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:14,524 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:14,524 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:14,524 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:14,524 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3404a93afc957274c1f21854ed231972 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-27 14:56:14,524 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. started... 2023-11-27 14:56:14,524 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. started... 2023-11-27 14:56:14,524 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 480bebe747409b1ccbbfa174b76957d7 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-27 14:56:14,524 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1cf135bbee78d983093ed681ad9bc867 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-27 14:56:14,540 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112733fae9f8daf14982b5efae285d9b80fe to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023112733fae9f8daf14982b5efae285d9b80fe 2023-11-27 14:56:14,942 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112736abd19abc364b848013587538311245 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023112736abd19abc364b848013587538311245 2023-11-27 14:56:14,942 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127c9d6ba2f44414397bf31c8a929aa2910 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231127c9d6ba2f44414397bf31c8a929aa2910 2023-11-27 14:56:14,947 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/0a7a1116ef2040efa491d346443cd79a 2023-11-27 14:56:14,971 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:56:14,971 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x41d5435c to 127.0.0.1:61098 2023-11-27 14:56:14,971 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:14,972 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:56:14,972 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1f518260 to 127.0.0.1:61098 2023-11-27 14:56:14,972 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:15,349 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/86955bcef07c4f52b5e12b315d49c5f3 2023-11-27 14:56:15,349 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/0da625f918984c6882342775dc98d9d9 2023-11-27 14:56:15,379 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127f298100577bd4217a3c23579b6a6388f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231127f298100577bd4217a3c23579b6a6388f 2023-11-27 14:56:15,379 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112776ae874f63ee4fc3a992fc9ffdf5eeb2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023112776ae874f63ee4fc3a992fc9ffdf5eeb2 2023-11-27 14:56:15,388 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/ff31618f465446b498192743ffdf65c6 2023-11-27 14:56:15,396 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/0a7a1116ef2040efa491d346443cd79a as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/0a7a1116ef2040efa491d346443cd79a 2023-11-27 14:56:15,400 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/0a7a1116ef2040efa491d346443cd79a, entries=4, sequenceid=9, filesize=5.4 K 2023-11-27 14:56:15,401 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/ff31618f465446b498192743ffdf65c6 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/ff31618f465446b498192743ffdf65c6 2023-11-27 14:56:15,405 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/ff31618f465446b498192743ffdf65c6, entries=2, sequenceid=9, filesize=5.1 K 2023-11-27 14:56:15,406 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 480bebe747409b1ccbbfa174b76957d7 in 882ms, sequenceid=9, compaction requested=false 2023-11-27 14:56:15,407 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:15,407 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:15,407 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:15,774 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112775640903de464d979d7274f115eb1550 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023112775640903de464d979d7274f115eb1550 2023-11-27 14:56:15,786 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/b4e10fb1fd80461086085787eabd5cc7 2023-11-27 14:56:15,790 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/0da625f918984c6882342775dc98d9d9 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0da625f918984c6882342775dc98d9d9 2023-11-27 14:56:15,794 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0da625f918984c6882342775dc98d9d9, entries=4, sequenceid=9, filesize=5.4 K 2023-11-27 14:56:15,795 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/b4e10fb1fd80461086085787eabd5cc7 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/b4e10fb1fd80461086085787eabd5cc7 2023-11-27 14:56:15,799 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/b4e10fb1fd80461086085787eabd5cc7, entries=2, sequenceid=9, filesize=5.1 K 2023-11-27 14:56:15,800 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 3404a93afc957274c1f21854ed231972 in 1276ms, sequenceid=9, compaction requested=false 2023-11-27 14:56:15,800 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:15,800 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:15,800 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:16,180 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/51c54ddc24cb454e8477352504b3782f 2023-11-27 14:56:16,184 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/86955bcef07c4f52b5e12b315d49c5f3 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/86955bcef07c4f52b5e12b315d49c5f3 2023-11-27 14:56:16,187 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/86955bcef07c4f52b5e12b315d49c5f3, entries=4, sequenceid=9, filesize=5.4 K 2023-11-27 14:56:16,188 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/51c54ddc24cb454e8477352504b3782f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/51c54ddc24cb454e8477352504b3782f 2023-11-27 14:56:16,192 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/51c54ddc24cb454e8477352504b3782f, entries=2, sequenceid=9, filesize=5.1 K 2023-11-27 14:56:16,193 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 1cf135bbee78d983093ed681ad9bc867 in 1669ms, sequenceid=9, compaction requested=false 2023-11-27 14:56:16,193 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:16,193 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:16,193 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:16,193 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:16,193 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,193 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-27 14:56:16,193 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-27 14:56:16,195 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,195 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:16,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:16,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:16,196 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:16,196 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:16,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:16,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:16,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:16,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:16,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-27 14:56:16,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f27a91a[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:16,198 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-27 14:56:16,198 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:16,200 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:16,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:16,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:16,201 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-27 14:56:16,201 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-27 14:56:16,201 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:16,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,201 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:16,202 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-27 14:56:16,202 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,202 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:16,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:16,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:16,202 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-27 14:56:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:16,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-27 14:56:16,205 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-27 14:56:16,205 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:16,205 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:16,205 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-27 14:56:16,205 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:16,206 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:16,206 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:16,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:16,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:16,206 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:16,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:16,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:16,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:16,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:16,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:16,207 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:16,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:16,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:16,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:16,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:16,208 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:16,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:16,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,211 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,211 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:16,211 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:16,211 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:16,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-27 14:56:16,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:16,211 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:16,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:16,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-27 14:56:16,211 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:16,211 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:16,212 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:16,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:16,211 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,212 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:16,212 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:16,212 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:16,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:16,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:16,312 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:16,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:16,314 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:16,314 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:16,314 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:16,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:16,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:16,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-27 14:56:16,318 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-27 14:56:16,318 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:16,318 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-27 14:56:16,318 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-27 14:56:16,319 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:16,319 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:16,321 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,321 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:16,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:16,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:16,321 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:16,321 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:16,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:16,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:16,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:16,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:16,322 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-27 14:56:16,322 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-27 14:56:16,322 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-27 14:56:16,322 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-27 14:56:16,322 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:16,323 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-27 14:56:16,323 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:16,323 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:16,323 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:16,323 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:16,323 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. started... 2023-11-27 14:56:16,323 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. started... 2023-11-27 14:56:16,323 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:16,323 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. started... 2023-11-27 14:56:16,323 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1cf135bbee78d983093ed681ad9bc867 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-27 14:56:16,323 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3404a93afc957274c1f21854ed231972 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-27 14:56:16,323 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 480bebe747409b1ccbbfa174b76957d7 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-27 14:56:16,334 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311271cb3e842d5ec4c2496b3114237575c1d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311271cb3e842d5ec4c2496b3114237575c1d 2023-11-27 14:56:16,734 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311271aca4ca224644ef3af58642dd33e651b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311271aca4ca224644ef3af58642dd33e651b 2023-11-27 14:56:16,735 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112781252330d2f34c62a530e327b91f5064 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023112781252330d2f34c62a530e327b91f5064 2023-11-27 14:56:16,744 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/75faea76a5e44d08b5aef2c34831f48e 2023-11-27 14:56:17,146 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/5a0506794300420fb1f08d926a7e4645 2023-11-27 14:56:17,146 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/a22876185d054afda7f40af2f63a0884 2023-11-27 14:56:17,163 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127cf733578077743fd8160abee94cd98f2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231127cf733578077743fd8160abee94cd98f2 2023-11-27 14:56:17,248 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:56:17,565 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311275547436febff40f7b91bd45699a62c06 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202311275547436febff40f7b91bd45699a62c06 2023-11-27 14:56:17,566 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127dacebe078f614aa5b208d40bddd7afa1 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231127dacebe078f614aa5b208d40bddd7afa1 2023-11-27 14:56:17,568 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/da25306ae964410184239565603b2a51 2023-11-27 14:56:17,572 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/75faea76a5e44d08b5aef2c34831f48e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/75faea76a5e44d08b5aef2c34831f48e 2023-11-27 14:56:17,577 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/75faea76a5e44d08b5aef2c34831f48e, entries=4, sequenceid=13, filesize=5.4 K 2023-11-27 14:56:17,578 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/da25306ae964410184239565603b2a51 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/da25306ae964410184239565603b2a51 2023-11-27 14:56:17,581 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/da25306ae964410184239565603b2a51, entries=2, sequenceid=13, filesize=5.1 K 2023-11-27 14:56:17,582 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 480bebe747409b1ccbbfa174b76957d7 in 1259ms, sequenceid=13, compaction requested=true 2023-11-27 14:56:17,582 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:17,582 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:17,582 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:17,591 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/f5fc098b8c6b4dd9b29d03babf52e9be 2023-11-27 14:56:17,599 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/5a0506794300420fb1f08d926a7e4645 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/5a0506794300420fb1f08d926a7e4645 2023-11-27 14:56:17,603 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/5a0506794300420fb1f08d926a7e4645, entries=4, sequenceid=13, filesize=5.4 K 2023-11-27 14:56:17,604 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/f5fc098b8c6b4dd9b29d03babf52e9be as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/f5fc098b8c6b4dd9b29d03babf52e9be 2023-11-27 14:56:17,608 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/f5fc098b8c6b4dd9b29d03babf52e9be, entries=2, sequenceid=13, filesize=5.1 K 2023-11-27 14:56:17,610 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 1cf135bbee78d983093ed681ad9bc867 in 1287ms, sequenceid=13, compaction requested=true 2023-11-27 14:56:17,610 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:17,610 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:17,610 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:17,991 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/4588009fad1f427880b8d1ea57475dd7 2023-11-27 14:56:17,995 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/a22876185d054afda7f40af2f63a0884 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/a22876185d054afda7f40af2f63a0884 2023-11-27 14:56:17,998 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/a22876185d054afda7f40af2f63a0884, entries=4, sequenceid=13, filesize=5.4 K 2023-11-27 14:56:17,999 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/4588009fad1f427880b8d1ea57475dd7 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/4588009fad1f427880b8d1ea57475dd7 2023-11-27 14:56:18,002 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/4588009fad1f427880b8d1ea57475dd7, entries=2, sequenceid=13, filesize=5.1 K 2023-11-27 14:56:18,003 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 3404a93afc957274c1f21854ed231972 in 1679ms, sequenceid=13, compaction requested=true 2023-11-27 14:56:18,003 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:18,003 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:18,003 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:18,003 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:18,003 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,003 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-27 14:56:18,003 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-27 14:56:18,004 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:18,004 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:18,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:18,005 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:18,005 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:18,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:18,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:18,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:18,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:18,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-27 14:56:18,007 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@328e6349[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:18,007 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-27 14:56:18,007 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:18,008 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:18,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:18,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:18,008 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-27 14:56:18,008 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-27 14:56:18,008 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,008 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:18,008 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:18,010 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-27 14:56:18,010 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,010 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:18,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:18,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:18,010 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-27 14:56:18,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:18,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:18,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:18,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:18,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:18,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:18,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-27 14:56:18,012 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-27 14:56:18,012 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:18,012 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:18,012 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-27 14:56:18,012 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:18,019 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:18,019 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:18,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:18,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:18,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:18,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:18,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:18,019 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:18,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:18,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:18,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:18,020 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:18,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:18,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:18,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,021 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:18,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:18,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,025 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,025 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:18,025 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:18,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:18,025 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:18,025 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:18,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:18,025 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-27 14:56:18,026 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:18,025 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:18,026 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,026 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:18,026 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:18,026 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:18,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:18,026 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:18,026 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:18,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:18,126 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:18,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:18,128 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:18,128 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:18,128 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:18,130 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:18,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:18,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-27 14:56:18,132 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-27 14:56:18,132 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:18,132 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-27 14:56:18,132 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-27 14:56:18,133 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:18,133 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:18,134 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,134 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:18,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:18,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:18,134 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:18,134 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:18,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:18,135 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:18,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:18,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:18,135 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-27 14:56:18,135 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-27 14:56:18,135 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-27 14:56:18,135 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-27 14:56:18,135 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:18,135 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-27 14:56:18,136 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:18,136 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. started... 2023-11-27 14:56:18,136 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:18,136 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:18,136 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3404a93afc957274c1f21854ed231972 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-27 14:56:18,136 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:18,136 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:18,136 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. started... 2023-11-27 14:56:18,136 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. started... 2023-11-27 14:56:18,136 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1cf135bbee78d983093ed681ad9bc867 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-27 14:56:18,136 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 480bebe747409b1ccbbfa174b76957d7 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-27 14:56:18,547 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311279f507fb77aad4d0ab1911d821e0d5ece to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311279f507fb77aad4d0ab1911d821e0d5ece 2023-11-27 14:56:18,547 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112767966e37de764a52853a4c082eabc89d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023112767966e37de764a52853a4c082eabc89d 2023-11-27 14:56:18,547 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311278969db65c9b146c19ec6ddface31f798 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311278969db65c9b146c19ec6ddface31f798 2023-11-27 14:56:18,952 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/cbc8a982752a495580c5e42474b40694 2023-11-27 14:56:18,953 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/6ffab0af0b7d4779ac9ebeab53b52005 2023-11-27 14:56:18,953 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/c5870a7729d54746b5dffa86423f360d 2023-11-27 14:56:18,970 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311271c0fbf932ca64064b737ebf18b345b9e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202311271c0fbf932ca64064b737ebf18b345b9e 2023-11-27 14:56:18,974 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/2553477794b64b5ba9a8ed8686bb7648 2023-11-27 14:56:18,978 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/c5870a7729d54746b5dffa86423f360d as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/c5870a7729d54746b5dffa86423f360d 2023-11-27 14:56:18,981 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/c5870a7729d54746b5dffa86423f360d, entries=4, sequenceid=17, filesize=5.4 K 2023-11-27 14:56:18,981 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/2553477794b64b5ba9a8ed8686bb7648 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/2553477794b64b5ba9a8ed8686bb7648 2023-11-27 14:56:18,984 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/2553477794b64b5ba9a8ed8686bb7648, entries=2, sequenceid=17, filesize=5.1 K 2023-11-27 14:56:18,985 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 1cf135bbee78d983093ed681ad9bc867 in 849ms, sequenceid=17, compaction requested=true 2023-11-27 14:56:18,985 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:18,985 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:18,985 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:19,371 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311277c3b832f94344180a9dd7e92648d0ed6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202311277c3b832f94344180a9dd7e92648d0ed6 2023-11-27 14:56:19,372 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127f83e0716780d4c168028bdbed42b3653 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231127f83e0716780d4c168028bdbed42b3653 2023-11-27 14:56:19,777 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/cf43913bcc8f442588415112a284e2f9 2023-11-27 14:56:19,777 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/c71b6378767e4684ac7e2862b3d08d9f 2023-11-27 14:56:19,781 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/6ffab0af0b7d4779ac9ebeab53b52005 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6ffab0af0b7d4779ac9ebeab53b52005 2023-11-27 14:56:19,781 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/cbc8a982752a495580c5e42474b40694 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/cbc8a982752a495580c5e42474b40694 2023-11-27 14:56:19,784 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6ffab0af0b7d4779ac9ebeab53b52005, entries=4, sequenceid=17, filesize=5.4 K 2023-11-27 14:56:19,785 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/cbc8a982752a495580c5e42474b40694, entries=4, sequenceid=17, filesize=5.4 K 2023-11-27 14:56:19,785 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/cf43913bcc8f442588415112a284e2f9 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/cf43913bcc8f442588415112a284e2f9 2023-11-27 14:56:19,785 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/c71b6378767e4684ac7e2862b3d08d9f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/c71b6378767e4684ac7e2862b3d08d9f 2023-11-27 14:56:19,789 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/cf43913bcc8f442588415112a284e2f9, entries=2, sequenceid=17, filesize=5.1 K 2023-11-27 14:56:19,789 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/c71b6378767e4684ac7e2862b3d08d9f, entries=2, sequenceid=17, filesize=5.1 K 2023-11-27 14:56:19,789 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 3404a93afc957274c1f21854ed231972 in 1653ms, sequenceid=17, compaction requested=true 2023-11-27 14:56:19,789 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 480bebe747409b1ccbbfa174b76957d7 in 1653ms, sequenceid=17, compaction requested=true 2023-11-27 14:56:19,789 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:19,790 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:19,790 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:19,790 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:19,790 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:19,790 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:19,790 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:19,790 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,790 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-27 14:56:19,790 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-27 14:56:19,792 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:19,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:19,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:19,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:19,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:19,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:19,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:19,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-27 14:56:19,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ad52689[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:19,794 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-27 14:56:19,794 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:19,795 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:19,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:19,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:19,795 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-27 14:56:19,795 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-27 14:56:19,795 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:19,795 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,795 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:19,796 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,796 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-27 14:56:19,796 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:19,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:19,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:19,796 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-27 14:56:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:19,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:19,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:19,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-27 14:56:19,799 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-27 14:56:19,799 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:19,799 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:19,799 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-27 14:56:19,799 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:19,800 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:19,800 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:19,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:19,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:19,800 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:19,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:19,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:19,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:19,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:19,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:19,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:19,801 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:19,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:19,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:19,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:19,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,802 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:19,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,805 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:19,806 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:19,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-27 14:56:19,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:19,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:19,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:19,806 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:19,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:19,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:19,906 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:19,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:19,916 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:19,924 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:19,932 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:19,933 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:19,966 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:19,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:19,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-27 14:56:19,968 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-27 14:56:19,968 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:19,968 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-27 14:56:19,968 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-27 14:56:19,969 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:19,969 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:19,970 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,970 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:19,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:19,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:19,970 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:19,970 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:19,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:19,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-27 14:56:19,976 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-27 14:56:19,976 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-27 14:56:19,976 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-27 14:56:19,976 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:19,976 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-27 14:56:19,977 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:19,977 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. started... 2023-11-27 14:56:19,977 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:19,977 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. started... 2023-11-27 14:56:19,977 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3404a93afc957274c1f21854ed231972 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-27 14:56:19,977 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 480bebe747409b1ccbbfa174b76957d7 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-27 14:56:19,979 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:19,979 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:19,979 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:19,979 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. started... 2023-11-27 14:56:19,979 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 1cf135bbee78d983093ed681ad9bc867 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-27 14:56:20,798 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/4083182b65944deabe717e8dddc3670a 2023-11-27 14:56:20,798 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/6a0d665335494ea38f2d8fdbc3359706 2023-11-27 14:56:20,798 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/b5756036046c43c299f4ba5a075a065a 2023-11-27 14:56:20,802 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4083182b65944deabe717e8dddc3670a 2023-11-27 14:56:20,802 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6a0d665335494ea38f2d8fdbc3359706 2023-11-27 14:56:20,802 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5756036046c43c299f4ba5a075a065a 2023-11-27 14:56:21,214 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/24a72f96cb5342d69088ba410494c952 2023-11-27 14:56:21,218 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24a72f96cb5342d69088ba410494c952 2023-11-27 14:56:21,219 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/b5756036046c43c299f4ba5a075a065a as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/b5756036046c43c299f4ba5a075a065a 2023-11-27 14:56:21,222 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5756036046c43c299f4ba5a075a065a 2023-11-27 14:56:21,222 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/b5756036046c43c299f4ba5a075a065a, entries=3, sequenceid=23, filesize=5.0 K 2023-11-27 14:56:21,223 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/24a72f96cb5342d69088ba410494c952 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/24a72f96cb5342d69088ba410494c952 2023-11-27 14:56:21,226 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24a72f96cb5342d69088ba410494c952 2023-11-27 14:56:21,226 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/24a72f96cb5342d69088ba410494c952, entries=1, sequenceid=23, filesize=5.0 K 2023-11-27 14:56:21,226 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 1cf135bbee78d983093ed681ad9bc867 in 1247ms, sequenceid=23, compaction requested=true 2023-11-27 14:56:21,226 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:21,226 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:21,227 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:21,615 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/7b23509ae6024ee3958671d88e05d553 2023-11-27 14:56:21,619 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b23509ae6024ee3958671d88e05d553 2023-11-27 14:56:21,619 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/15584a9834b4437b994b850f70044cc4 2023-11-27 14:56:21,619 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/4083182b65944deabe717e8dddc3670a as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/4083182b65944deabe717e8dddc3670a 2023-11-27 14:56:21,622 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 15584a9834b4437b994b850f70044cc4 2023-11-27 14:56:21,623 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/6a0d665335494ea38f2d8fdbc3359706 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6a0d665335494ea38f2d8fdbc3359706 2023-11-27 14:56:21,623 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4083182b65944deabe717e8dddc3670a 2023-11-27 14:56:21,623 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/4083182b65944deabe717e8dddc3670a, entries=3, sequenceid=23, filesize=5.0 K 2023-11-27 14:56:21,624 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/7b23509ae6024ee3958671d88e05d553 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/7b23509ae6024ee3958671d88e05d553 2023-11-27 14:56:21,626 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6a0d665335494ea38f2d8fdbc3359706 2023-11-27 14:56:21,626 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6a0d665335494ea38f2d8fdbc3359706, entries=3, sequenceid=23, filesize=5.0 K 2023-11-27 14:56:21,627 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/15584a9834b4437b994b850f70044cc4 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/15584a9834b4437b994b850f70044cc4 2023-11-27 14:56:21,627 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b23509ae6024ee3958671d88e05d553 2023-11-27 14:56:21,627 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/7b23509ae6024ee3958671d88e05d553, entries=1, sequenceid=23, filesize=5.0 K 2023-11-27 14:56:21,628 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 480bebe747409b1ccbbfa174b76957d7 in 1651ms, sequenceid=23, compaction requested=true 2023-11-27 14:56:21,628 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:21,628 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:21,628 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:21,630 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 15584a9834b4437b994b850f70044cc4 2023-11-27 14:56:21,630 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/15584a9834b4437b994b850f70044cc4, entries=1, sequenceid=23, filesize=5.0 K 2023-11-27 14:56:21,631 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 3404a93afc957274c1f21854ed231972 in 1654ms, sequenceid=23, compaction requested=true 2023-11-27 14:56:21,631 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:21,631 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:21,631 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:21,631 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:21,631 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,631 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-27 14:56:21,631 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-27 14:56:21,632 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,632 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:21,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:21,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:21,633 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:21,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:21,633 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:21,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:21,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:21,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:21,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-27 14:56:21,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5694daef[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:21,634 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-27 14:56:21,634 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:21,635 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:21,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:21,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:21,635 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-27 14:56:21,635 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-27 14:56:21,635 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:21,635 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,635 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:21,641 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,641 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-27 14:56:21,641 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:21,641 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-27 14:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:21,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:21,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:21,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:21,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:21,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:21,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-27 14:56:21,643 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-27 14:56:21,643 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:21,643 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:21,643 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-27 14:56:21,643 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:21,645 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:21,645 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:21,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:21,645 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:21,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:21,645 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:21,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:21,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:21,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:21,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:21,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,646 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:21,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-27 14:56:21,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,648 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,648 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:21,649 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:21,649 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-27 14:56:21,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:21,649 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:21,649 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:21,649 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-27 14:56:21,649 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:21,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:21,649 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:21,649 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:21,649 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-27 14:56:21,649 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-27 14:56:21,649 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:21,649 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:21,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:21,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:21,749 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:21,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:21,750 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:21,750 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): 3404a93afc957274c1f21854ed231972/family1 is initiating major compaction (all files) 2023-11-27 14:56:21,751 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:21,751 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:21,751 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of 3404a93afc957274c1f21854ed231972/family1 in ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:21,752 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/f6a1d73e384249a888947261aa58c919, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0da625f918984c6882342775dc98d9d9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/a22876185d054afda7f40af2f63a0884, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6ffab0af0b7d4779ac9ebeab53b52005, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6a0d665335494ea38f2d8fdbc3359706] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp, totalSize=26.6 K 2023-11-27 14:56:21,752 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting f6a1d73e384249a888947261aa58c919, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096972704 2023-11-27 14:56:21,752 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 0da625f918984c6882342775dc98d9d9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096974515 2023-11-27 14:56:21,753 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting a22876185d054afda7f40af2f63a0884, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096976314 2023-11-27 14:56:21,753 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 6ffab0af0b7d4779ac9ebeab53b52005, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096978128 2023-11-27 14:56:21,753 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 6a0d665335494ea38f2d8fdbc3359706, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:21,762 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): 3404a93afc957274c1f21854ed231972#family1#compaction#148 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:22,170 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311273cb85a2f17734fec91628ad4a6092898_del 2023-11-27 14:56:22,170 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311273cb85a2f17734fec91628ad4a6092898_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311273cb85a2f17734fec91628ad4a6092898_del 2023-11-27 14:56:22,983 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f566a36c1d8046a69ba560452aa280cb 2023-11-27 14:56:22,984 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family1/f566a36c1d8046a69ba560452aa280cb as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/f566a36c1d8046a69ba560452aa280cb 2023-11-27 14:56:22,987 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f566a36c1d8046a69ba560452aa280cb 2023-11-27 14:56:22,988 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3404a93afc957274c1f21854ed231972/family1 of 3404a93afc957274c1f21854ed231972 into f566a36c1d8046a69ba560452aa280cb(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:22,988 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:22,988 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:22,988 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): 3404a93afc957274c1f21854ed231972/family2 is initiating major compaction (all files) 2023-11-27 14:56:22,988 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:22,988 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:22,989 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of 3404a93afc957274c1f21854ed231972/family2 in ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:22,989 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/9e15311d2d9b44e29b1d36b369a11fc3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/b4e10fb1fd80461086085787eabd5cc7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/4588009fad1f427880b8d1ea57475dd7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/cf43913bcc8f442588415112a284e2f9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/15584a9834b4437b994b850f70044cc4] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp, totalSize=25.3 K 2023-11-27 14:56:22,989 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 9e15311d2d9b44e29b1d36b369a11fc3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096972704 2023-11-27 14:56:22,990 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting b4e10fb1fd80461086085787eabd5cc7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096974515 2023-11-27 14:56:22,990 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 4588009fad1f427880b8d1ea57475dd7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096976314 2023-11-27 14:56:22,990 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting cf43913bcc8f442588415112a284e2f9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096978128 2023-11-27 14:56:22,991 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 15584a9834b4437b994b850f70044cc4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:23,000 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): 3404a93afc957274c1f21854ed231972#family2#compaction#149 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:23,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/f6a1d73e384249a888947261aa58c919, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0da625f918984c6882342775dc98d9d9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/a22876185d054afda7f40af2f63a0884, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6ffab0af0b7d4779ac9ebeab53b52005, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6a0d665335494ea38f2d8fdbc3359706] to archive 2023-11-27 14:56:23,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:23,053 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/f6a1d73e384249a888947261aa58c919 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/f6a1d73e384249a888947261aa58c919 2023-11-27 14:56:23,054 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0da625f918984c6882342775dc98d9d9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0da625f918984c6882342775dc98d9d9 2023-11-27 14:56:23,055 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/a22876185d054afda7f40af2f63a0884 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/a22876185d054afda7f40af2f63a0884 2023-11-27 14:56:23,055 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6ffab0af0b7d4779ac9ebeab53b52005 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6ffab0af0b7d4779ac9ebeab53b52005 2023-11-27 14:56:23,056 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6a0d665335494ea38f2d8fdbc3359706 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/6a0d665335494ea38f2d8fdbc3359706 2023-11-27 14:56:23,407 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231127ce9839ac478b41a0a18e9af4bfa44f2c_del 2023-11-27 14:56:23,408 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127ce9839ac478b41a0a18e9af4bfa44f2c_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231127ce9839ac478b41a0a18e9af4bfa44f2c_del 2023-11-27 14:56:24,220 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9f63ccbab2564cac9c51068424959f47 2023-11-27 14:56:24,221 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/.tmp/family2/9f63ccbab2564cac9c51068424959f47 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/9f63ccbab2564cac9c51068424959f47 2023-11-27 14:56:24,224 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9f63ccbab2564cac9c51068424959f47 2023-11-27 14:56:24,225 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3404a93afc957274c1f21854ed231972/family2 of 3404a93afc957274c1f21854ed231972 into 9f63ccbab2564cac9c51068424959f47(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:24,225 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:24,225 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:24,226 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): 480bebe747409b1ccbbfa174b76957d7/family1 is initiating major compaction (all files) 2023-11-27 14:56:24,226 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:24,226 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:24,226 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of 480bebe747409b1ccbbfa174b76957d7/family1 in ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:24,226 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/59544ee789414c46addbe33bdf3065be, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/0a7a1116ef2040efa491d346443cd79a, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/75faea76a5e44d08b5aef2c34831f48e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/cbc8a982752a495580c5e42474b40694, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/4083182b65944deabe717e8dddc3670a] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp, totalSize=26.6 K 2023-11-27 14:56:24,226 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 59544ee789414c46addbe33bdf3065be, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096972705 2023-11-27 14:56:24,227 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 0a7a1116ef2040efa491d346443cd79a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096974515 2023-11-27 14:56:24,227 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 75faea76a5e44d08b5aef2c34831f48e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096976314 2023-11-27 14:56:24,227 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting cbc8a982752a495580c5e42474b40694, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096978128 2023-11-27 14:56:24,228 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 4083182b65944deabe717e8dddc3670a, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:24,237 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): 480bebe747409b1ccbbfa174b76957d7#family1#compaction#150 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:24,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/9e15311d2d9b44e29b1d36b369a11fc3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/b4e10fb1fd80461086085787eabd5cc7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/4588009fad1f427880b8d1ea57475dd7, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/cf43913bcc8f442588415112a284e2f9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/15584a9834b4437b994b850f70044cc4] to archive 2023-11-27 14:56:24,250 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:24,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/9e15311d2d9b44e29b1d36b369a11fc3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/9e15311d2d9b44e29b1d36b369a11fc3 2023-11-27 14:56:24,253 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/b4e10fb1fd80461086085787eabd5cc7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/b4e10fb1fd80461086085787eabd5cc7 2023-11-27 14:56:24,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/4588009fad1f427880b8d1ea57475dd7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/4588009fad1f427880b8d1ea57475dd7 2023-11-27 14:56:24,255 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/cf43913bcc8f442588415112a284e2f9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/cf43913bcc8f442588415112a284e2f9 2023-11-27 14:56:24,256 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/15584a9834b4437b994b850f70044cc4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family2/15584a9834b4437b994b850f70044cc4 2023-11-27 14:56:24,645 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311271ae9021960934b93b182121e1a10300e_del 2023-11-27 14:56:24,645 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311271ae9021960934b93b182121e1a10300e_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311271ae9021960934b93b182121e1a10300e_del 2023-11-27 14:56:25,458 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fefe604b45d940b8854f67b93f047ce9 2023-11-27 14:56:25,458 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family1/fefe604b45d940b8854f67b93f047ce9 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/fefe604b45d940b8854f67b93f047ce9 2023-11-27 14:56:25,462 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fefe604b45d940b8854f67b93f047ce9 2023-11-27 14:56:25,463 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 480bebe747409b1ccbbfa174b76957d7/family1 of 480bebe747409b1ccbbfa174b76957d7 into fefe604b45d940b8854f67b93f047ce9(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:25,463 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:25,463 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:25,463 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): 480bebe747409b1ccbbfa174b76957d7/family2 is initiating major compaction (all files) 2023-11-27 14:56:25,463 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:25,463 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:25,464 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of 480bebe747409b1ccbbfa174b76957d7/family2 in ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:25,464 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/9878a9d04c334bf993b75c9b88d055e6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/ff31618f465446b498192743ffdf65c6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/da25306ae964410184239565603b2a51, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/c71b6378767e4684ac7e2862b3d08d9f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/7b23509ae6024ee3958671d88e05d553] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp, totalSize=25.3 K 2023-11-27 14:56:25,464 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 9878a9d04c334bf993b75c9b88d055e6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096972705 2023-11-27 14:56:25,465 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting ff31618f465446b498192743ffdf65c6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096974515 2023-11-27 14:56:25,465 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting da25306ae964410184239565603b2a51, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096976314 2023-11-27 14:56:25,465 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting c71b6378767e4684ac7e2862b3d08d9f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096978128 2023-11-27 14:56:25,465 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 7b23509ae6024ee3958671d88e05d553, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:25,473 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): 480bebe747409b1ccbbfa174b76957d7#family2#compaction#151 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:25,549 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/59544ee789414c46addbe33bdf3065be, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/0a7a1116ef2040efa491d346443cd79a, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/75faea76a5e44d08b5aef2c34831f48e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/cbc8a982752a495580c5e42474b40694, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/4083182b65944deabe717e8dddc3670a] to archive 2023-11-27 14:56:25,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:25,551 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/59544ee789414c46addbe33bdf3065be to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/59544ee789414c46addbe33bdf3065be 2023-11-27 14:56:25,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/0a7a1116ef2040efa491d346443cd79a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/0a7a1116ef2040efa491d346443cd79a 2023-11-27 14:56:25,553 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/75faea76a5e44d08b5aef2c34831f48e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/75faea76a5e44d08b5aef2c34831f48e 2023-11-27 14:56:25,554 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/cbc8a982752a495580c5e42474b40694 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/cbc8a982752a495580c5e42474b40694 2023-11-27 14:56:25,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/4083182b65944deabe717e8dddc3670a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/4083182b65944deabe717e8dddc3670a 2023-11-27 14:56:25,880 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231127ffb59e55f9de4859b22f0e8d7f6b7ccf_del 2023-11-27 14:56:25,880 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127ffb59e55f9de4859b22f0e8d7f6b7ccf_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231127ffb59e55f9de4859b22f0e8d7f6b7ccf_del 2023-11-27 14:56:26,693 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for daf5f72db71b4408bd2f8a4060781e01 2023-11-27 14:56:26,694 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/.tmp/family2/daf5f72db71b4408bd2f8a4060781e01 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/daf5f72db71b4408bd2f8a4060781e01 2023-11-27 14:56:26,698 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for daf5f72db71b4408bd2f8a4060781e01 2023-11-27 14:56:26,699 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 480bebe747409b1ccbbfa174b76957d7/family2 of 480bebe747409b1ccbbfa174b76957d7 into daf5f72db71b4408bd2f8a4060781e01(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:26,699 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:26,699 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:26,699 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): 1cf135bbee78d983093ed681ad9bc867/family1 is initiating major compaction (all files) 2023-11-27 14:56:26,699 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:26,699 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:26,699 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of 1cf135bbee78d983093ed681ad9bc867/family1 in ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:26,699 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/2dd63e9a981e4c969641bc1cefa98030, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/86955bcef07c4f52b5e12b315d49c5f3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/5a0506794300420fb1f08d926a7e4645, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/c5870a7729d54746b5dffa86423f360d, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/b5756036046c43c299f4ba5a075a065a] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp, totalSize=26.6 K 2023-11-27 14:56:26,700 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 2dd63e9a981e4c969641bc1cefa98030, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096972705 2023-11-27 14:56:26,700 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 86955bcef07c4f52b5e12b315d49c5f3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096974515 2023-11-27 14:56:26,700 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 5a0506794300420fb1f08d926a7e4645, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096976314 2023-11-27 14:56:26,701 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting c5870a7729d54746b5dffa86423f360d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096978128 2023-11-27 14:56:26,701 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting b5756036046c43c299f4ba5a075a065a, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:26,710 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): 1cf135bbee78d983093ed681ad9bc867#family1#compaction#152 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:26,718 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311279427bdf8efeb4011b29c5c3e63af3697_del 2023-11-27 14:56:26,718 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311279427bdf8efeb4011b29c5c3e63af3697_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311279427bdf8efeb4011b29c5c3e63af3697_del 2023-11-27 14:56:26,750 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/9878a9d04c334bf993b75c9b88d055e6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/ff31618f465446b498192743ffdf65c6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/da25306ae964410184239565603b2a51, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/c71b6378767e4684ac7e2862b3d08d9f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/7b23509ae6024ee3958671d88e05d553] to archive 2023-11-27 14:56:26,751 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:26,753 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/9878a9d04c334bf993b75c9b88d055e6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/9878a9d04c334bf993b75c9b88d055e6 2023-11-27 14:56:26,754 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/ff31618f465446b498192743ffdf65c6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/ff31618f465446b498192743ffdf65c6 2023-11-27 14:56:26,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/da25306ae964410184239565603b2a51 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/da25306ae964410184239565603b2a51 2023-11-27 14:56:26,757 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/c71b6378767e4684ac7e2862b3d08d9f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/c71b6378767e4684ac7e2862b3d08d9f 2023-11-27 14:56:26,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/7b23509ae6024ee3958671d88e05d553 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family2/7b23509ae6024ee3958671d88e05d553 2023-11-27 14:56:27,532 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cac83b39abb4bc7b8369fd2aff581a7 2023-11-27 14:56:27,533 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family1/0cac83b39abb4bc7b8369fd2aff581a7 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/0cac83b39abb4bc7b8369fd2aff581a7 2023-11-27 14:56:27,537 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0cac83b39abb4bc7b8369fd2aff581a7 2023-11-27 14:56:27,538 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1cf135bbee78d983093ed681ad9bc867/family1 of 1cf135bbee78d983093ed681ad9bc867 into 0cac83b39abb4bc7b8369fd2aff581a7(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:27,538 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:27,538 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:27,538 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): 1cf135bbee78d983093ed681ad9bc867/family2 is initiating major compaction (all files) 2023-11-27 14:56:27,538 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:27,538 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:27,538 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of 1cf135bbee78d983093ed681ad9bc867/family2 in ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:27,539 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/fccdfeb0b1a846e393e1178e004dbb50, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/51c54ddc24cb454e8477352504b3782f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/f5fc098b8c6b4dd9b29d03babf52e9be, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/2553477794b64b5ba9a8ed8686bb7648, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/24a72f96cb5342d69088ba410494c952] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp, totalSize=25.3 K 2023-11-27 14:56:27,539 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting fccdfeb0b1a846e393e1178e004dbb50, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701096972705 2023-11-27 14:56:27,539 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 51c54ddc24cb454e8477352504b3782f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701096974515 2023-11-27 14:56:27,539 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting f5fc098b8c6b4dd9b29d03babf52e9be, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701096976314 2023-11-27 14:56:27,540 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 2553477794b64b5ba9a8ed8686bb7648, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701096978128 2023-11-27 14:56:27,540 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 24a72f96cb5342d69088ba410494c952, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:27,548 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): 1cf135bbee78d983093ed681ad9bc867#family2#compaction#153 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:27,549 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/2dd63e9a981e4c969641bc1cefa98030, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/86955bcef07c4f52b5e12b315d49c5f3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/5a0506794300420fb1f08d926a7e4645, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/c5870a7729d54746b5dffa86423f360d, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/b5756036046c43c299f4ba5a075a065a] to archive 2023-11-27 14:56:27,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:27,551 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/2dd63e9a981e4c969641bc1cefa98030 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/2dd63e9a981e4c969641bc1cefa98030 2023-11-27 14:56:27,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/86955bcef07c4f52b5e12b315d49c5f3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/86955bcef07c4f52b5e12b315d49c5f3 2023-11-27 14:56:27,553 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/5a0506794300420fb1f08d926a7e4645 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/5a0506794300420fb1f08d926a7e4645 2023-11-27 14:56:27,554 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/c5870a7729d54746b5dffa86423f360d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/c5870a7729d54746b5dffa86423f360d 2023-11-27 14:56:27,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/b5756036046c43c299f4ba5a075a065a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/b5756036046c43c299f4ba5a075a065a 2023-11-27 14:56:27,956 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231127607e55330908443a870a3d974197f615_del 2023-11-27 14:56:27,956 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127607e55330908443a870a3d974197f615_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231127607e55330908443a870a3d974197f615_del 2023-11-27 14:56:28,369 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 11e95c05f12b41129526d96d93ba3bcd 2023-11-27 14:56:28,370 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/.tmp/family2/11e95c05f12b41129526d96d93ba3bcd as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/11e95c05f12b41129526d96d93ba3bcd 2023-11-27 14:56:28,374 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 11e95c05f12b41129526d96d93ba3bcd 2023-11-27 14:56:28,375 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1cf135bbee78d983093ed681ad9bc867/family2 of 1cf135bbee78d983093ed681ad9bc867 into 11e95c05f12b41129526d96d93ba3bcd(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:28,375 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:28,378 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:28,381 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:28,384 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:28,384 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:28,389 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-27 14:56:28,403 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-11-27 14:56:28,407 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311273cb85a2f17734fec91628ad4a6092898_del 2023-11-27 14:56:28,410 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311279427bdf8efeb4011b29c5c3e63af3697_del 2023-11-27 14:56:28,413 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311271ae9021960934b93b182121e1a10300e_del 2023-11-27 14:56:28,413 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-11-27 14:56:28,414 DEBUG [Listener at localhost/37439] zookeeper.ReadOnlyZKClient(139): Connect 0x64b8e4ef to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:56:28,419 DEBUG [Listener at localhost/37439] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5fb052e2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:56:28,420 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231127 for table='ns:testMinorCompaction' and column='family1' 2023-11-27 14:56:28,420 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231127 for table='ns:testMinorCompaction' and column='family1' 2023-11-27 14:56:28,420 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231127 for table='ns:testMinorCompaction' and column='family1' 2023-11-27 14:56:28,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/fccdfeb0b1a846e393e1178e004dbb50, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/51c54ddc24cb454e8477352504b3782f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/f5fc098b8c6b4dd9b29d03babf52e9be, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/2553477794b64b5ba9a8ed8686bb7648, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/24a72f96cb5342d69088ba410494c952] to archive 2023-11-27 14:56:28,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:28,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/fccdfeb0b1a846e393e1178e004dbb50 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/fccdfeb0b1a846e393e1178e004dbb50 2023-11-27 14:56:28,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/51c54ddc24cb454e8477352504b3782f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/51c54ddc24cb454e8477352504b3782f 2023-11-27 14:56:28,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/f5fc098b8c6b4dd9b29d03babf52e9be to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/f5fc098b8c6b4dd9b29d03babf52e9be 2023-11-27 14:56:28,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/2553477794b64b5ba9a8ed8686bb7648 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/2553477794b64b5ba9a8ed8686bb7648 2023-11-27 14:56:28,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/24a72f96cb5342d69088ba410494c952 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family2/24a72f96cb5342d69088ba410494c952 2023-11-27 14:56:28,857 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311271fa8b1d206424227aeb77b457a02309a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311271fa8b1d206424227aeb77b457a02309a 2023-11-27 14:56:28,858 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311277ee6fb41d7894378a067872772f6fde0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311277ee6fb41d7894378a067872772f6fde0 2023-11-27 14:56:28,858 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='ns:testMinorCompaction' column='family1' 2023-11-27 14:56:28,859 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='ns:testMinorCompaction' column='family1' 2023-11-27 14:56:28,877 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:56:28,879 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41722, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:56:28,885 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:56:28,888 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35038, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:56:28,901 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231127/family1/d422a74abe1f486f9a7c1619a1915ad5 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-11-27 14:56:28,902 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231127/family1/d422a74abe1f486f9a7c1619a1915ad5}] 2023-11-27 14:56:28,902 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231127/family1/34178f65cf624b368d5ca178dd744aad first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-11-27 14:56:28,902 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231127/family1/34178f65cf624b368d5ca178dd744aad}] 2023-11-27 14:56:28,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231127/family1/34178f65cf624b368d5ca178dd744aad for inclusion in 480bebe747409b1ccbbfa174b76957d7/family1 2023-11-27 14:56:28,910 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231127/family1/d422a74abe1f486f9a7c1619a1915ad5 for inclusion in 3404a93afc957274c1f21854ed231972/family1 2023-11-27 14:56:28,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-11-27 14:56:28,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-11-27 14:56:28,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-27 14:56:28,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last=1 2023-11-27 14:56:28,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:28,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:28,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231127/family1/34178f65cf624b368d5ca178dd744aad to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__62hp1jlv0717roaha71lujonqtm4ah64qjpma48o69ci8m4h62og0se316nu4595/family1/34178f65cf624b368d5ca178dd744aad 2023-11-27 14:56:28,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231127/family1/d422a74abe1f486f9a7c1619a1915ad5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__a6dutk9fp1aeats6tp4qb6um4t6tev3e9bkvdt0hcp7jf2o894oapf6vv4r6ecos/family1/d422a74abe1f486f9a7c1619a1915ad5 2023-11-27 14:56:28,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__62hp1jlv0717roaha71lujonqtm4ah64qjpma48o69ci8m4h62og0se316nu4595/family1/34178f65cf624b368d5ca178dd744aad as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/23c1ec51a1de428c8cc28348915553ac_SeqId_28_ 2023-11-27 14:56:28,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__a6dutk9fp1aeats6tp4qb6um4t6tev3e9bkvdt0hcp7jf2o894oapf6vv4r6ecos/family1/d422a74abe1f486f9a7c1619a1915ad5 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0d68875ce71a4c99901e32e0ae5da12c_SeqId_28_ 2023-11-27 14:56:28,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__62hp1jlv0717roaha71lujonqtm4ah64qjpma48o69ci8m4h62og0se316nu4595/family1/34178f65cf624b368d5ca178dd744aad into 480bebe747409b1ccbbfa174b76957d7/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/23c1ec51a1de428c8cc28348915553ac_SeqId_28_ - updating store file list. 2023-11-27 14:56:28,916 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__a6dutk9fp1aeats6tp4qb6um4t6tev3e9bkvdt0hcp7jf2o894oapf6vv4r6ecos/family1/d422a74abe1f486f9a7c1619a1915ad5 into 3404a93afc957274c1f21854ed231972/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0d68875ce71a4c99901e32e0ae5da12c_SeqId_28_ - updating store file list. 2023-11-27 14:56:28,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/23c1ec51a1de428c8cc28348915553ac_SeqId_28_ into 480bebe747409b1ccbbfa174b76957d7/family1 2023-11-27 14:56:28,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__62hp1jlv0717roaha71lujonqtm4ah64qjpma48o69ci8m4h62og0se316nu4595/family1/34178f65cf624b368d5ca178dd744aad into 480bebe747409b1ccbbfa174b76957d7/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/family1/23c1ec51a1de428c8cc28348915553ac_SeqId_28_) 2023-11-27 14:56:28,919 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0d68875ce71a4c99901e32e0ae5da12c_SeqId_28_ into 3404a93afc957274c1f21854ed231972/family1 2023-11-27 14:56:28,919 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__a6dutk9fp1aeats6tp4qb6um4t6tev3e9bkvdt0hcp7jf2o894oapf6vv4r6ecos/family1/d422a74abe1f486f9a7c1619a1915ad5 into 3404a93afc957274c1f21854ed231972/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/family1/0d68875ce71a4c99901e32e0ae5da12c_SeqId_28_) 2023-11-27 14:56:28,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__62hp1jlv0717roaha71lujonqtm4ah64qjpma48o69ci8m4h62og0se316nu4595/family1/34178f65cf624b368d5ca178dd744aad 2023-11-27 14:56:28,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__a6dutk9fp1aeats6tp4qb6um4t6tev3e9bkvdt0hcp7jf2o894oapf6vv4r6ecos/family1/d422a74abe1f486f9a7c1619a1915ad5 2023-11-27 14:56:28,922 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='ns:testMinorCompaction' column='family1' 2023-11-27 14:56:28,922 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='ns:testMinorCompaction' column='family1' 2023-11-27 14:56:28,922 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:28,922 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:28,923 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231127beca75b363da4fcd8645640e4343a3e5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231127beca75b363da4fcd8645640e4343a3e5 2023-11-27 14:56:28,924 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-27 14:56:28,924 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231127' 2023-11-27 14:56:28,924 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231127f9364d5504ac48c2afce1337e3bc0eb7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231127f9364d5504ac48c2afce1337e3bc0eb7 2023-11-27 14:56:28,924 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-27 14:56:28,924 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231127' 2023-11-27 14:56:28,924 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231127 for table='ns:testMinorCompaction' and column='family1' 2023-11-27 14:56:28,924 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231127 for table='ns:testMinorCompaction' and column='family1' 2023-11-27 14:56:29,263 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127c5be0317478f49588c2fe00fd065b736 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231127c5be0317478f49588c2fe00fd065b736 2023-11-27 14:56:29,264 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231127 table='ns:testMinorCompaction' column='family1' 2023-11-27 14:56:29,290 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231127/family1/aa64952965e746ea9a5c32da50e05976 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-11-27 14:56:29,290 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231127/family1/aa64952965e746ea9a5c32da50e05976}] 2023-11-27 14:56:29,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231127/family1/aa64952965e746ea9a5c32da50e05976 for inclusion in 1cf135bbee78d983093ed681ad9bc867/family1 2023-11-27 14:56:29,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-11-27 14:56:29,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-27 14:56:29,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:29,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231127/family1/aa64952965e746ea9a5c32da50e05976 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__aehfndrrn4squin16vnpis4osqq1njo3i2tqh20c6nvchii8glviad650c1grs/family1/aa64952965e746ea9a5c32da50e05976 2023-11-27 14:56:29,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__aehfndrrn4squin16vnpis4osqq1njo3i2tqh20c6nvchii8glviad650c1grs/family1/aa64952965e746ea9a5c32da50e05976 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/d9dc899106e54973a1f409fe293b4011_SeqId_28_ 2023-11-27 14:56:29,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__aehfndrrn4squin16vnpis4osqq1njo3i2tqh20c6nvchii8glviad650c1grs/family1/aa64952965e746ea9a5c32da50e05976 into 1cf135bbee78d983093ed681ad9bc867/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/d9dc899106e54973a1f409fe293b4011_SeqId_28_ - updating store file list. 2023-11-27 14:56:29,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/d9dc899106e54973a1f409fe293b4011_SeqId_28_ into 1cf135bbee78d983093ed681ad9bc867/family1 2023-11-27 14:56:29,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__aehfndrrn4squin16vnpis4osqq1njo3i2tqh20c6nvchii8glviad650c1grs/family1/aa64952965e746ea9a5c32da50e05976 into 1cf135bbee78d983093ed681ad9bc867/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/family1/d9dc899106e54973a1f409fe293b4011_SeqId_28_) 2023-11-27 14:56:29,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__ns_testMinorCompaction__aehfndrrn4squin16vnpis4osqq1njo3i2tqh20c6nvchii8glviad650c1grs/family1/aa64952965e746ea9a5c32da50e05976 2023-11-27 14:56:29,303 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231127 table='ns:testMinorCompaction' column='family1' 2023-11-27 14:56:29,303 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:29,304 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311278a8c5e40f9b1419fa3c70bd2ec0d9c6c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311278a8c5e40f9b1419fa3c70bd2ec0d9c6c 2023-11-27 14:56:29,304 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-27 14:56:29,304 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231127' 2023-11-27 14:56:29,305 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231127 for table='ns:testMinorCompaction' and column='family1' 2023-11-27 14:56:29,305 INFO [Listener at localhost/37439] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-11-27 14:56:29,307 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:29,309 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:29,311 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:29,312 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:29,331 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=254 (was 289), OpenFileDescriptor=662 (was 790), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=155 (was 182), ProcessCount=168 (was 168), AvailableMemoryMB=8018 (was 8433) 2023-11-27 14:56:29,344 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=254, OpenFileDescriptor=662, MaxFileDescriptor=60000, SystemLoadAverage=155, ProcessCount=168, AvailableMemoryMB=8018 2023-11-27 14:56:29,345 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-27 14:56:29,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:29,347 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:56:29,347 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-11-27 14:56:29,348 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:56:29,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-27 14:56:29,349 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:29,350 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231 empty. 2023-11-27 14:56:29,350 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:29,350 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-11-27 14:56:29,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-27 14:56:29,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-27 14:56:29,757 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-27 14:56:29,758 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => d7e7b1527d4ce274f466396dcf7e3231, NAME => 'testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:56:29,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-27 14:56:30,163 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:30,163 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing d7e7b1527d4ce274f466396dcf7e3231, disabling compactions & flushes 2023-11-27 14:56:30,163 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:30,163 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:30,163 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. after waiting 0 ms 2023-11-27 14:56:30,163 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:30,163 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:30,163 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:30,165 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:56:30,166 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701096990166"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701096990166"}]},"ts":"1701096990166"} 2023-11-27 14:56:30,167 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-27 14:56:30,168 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:56:30,168 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096990168"}]},"ts":"1701096990168"} 2023-11-27 14:56:30,169 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-11-27 14:56:30,173 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d7e7b1527d4ce274f466396dcf7e3231, ASSIGN}] 2023-11-27 14:56:30,175 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d7e7b1527d4ce274f466396dcf7e3231, ASSIGN 2023-11-27 14:56:30,175 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d7e7b1527d4ce274f466396dcf7e3231, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:56:30,326 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d7e7b1527d4ce274f466396dcf7e3231, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:30,327 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701096990326"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701096990326"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701096990326"}]},"ts":"1701096990326"} 2023-11-27 14:56:30,328 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure d7e7b1527d4ce274f466396dcf7e3231, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:30,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-27 14:56:30,480 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:30,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:30,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d7e7b1527d4ce274f466396dcf7e3231, NAME => 'testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231.', STARTKEY => '', ENDKEY => ''} 2023-11-27 14:56:30,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:30,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:30,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:30,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:30,486 INFO [StoreOpener-d7e7b1527d4ce274f466396dcf7e3231-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:30,487 DEBUG [StoreOpener-d7e7b1527d4ce274f466396dcf7e3231-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1 2023-11-27 14:56:30,487 DEBUG [StoreOpener-d7e7b1527d4ce274f466396dcf7e3231-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1 2023-11-27 14:56:30,487 INFO [StoreOpener-d7e7b1527d4ce274f466396dcf7e3231-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d7e7b1527d4ce274f466396dcf7e3231 columnFamilyName family1 2023-11-27 14:56:30,489 INFO [StoreOpener-d7e7b1527d4ce274f466396dcf7e3231-1] regionserver.HStore(310): Store=d7e7b1527d4ce274f466396dcf7e3231/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:30,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:30,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:30,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:30,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:56:30,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d7e7b1527d4ce274f466396dcf7e3231; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66824004, jitterRate=-0.004244744777679443}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-27 14:56:30,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:30,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231., pid=114, masterSystemTime=1701096990480 2023-11-27 14:56:30,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:30,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:30,496 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d7e7b1527d4ce274f466396dcf7e3231, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:30,496 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1701096990496"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701096990496"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701096990496"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701096990496"}]},"ts":"1701096990496"} 2023-11-27 14:56:30,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=114, resume processing ppid=113 2023-11-27 14:56:30,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure d7e7b1527d4ce274f466396dcf7e3231, server=jenkins-hbase4.apache.org,42669,1701096850360 in 170 msec 2023-11-27 14:56:30,501 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=113, resume processing ppid=112 2023-11-27 14:56:30,501 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d7e7b1527d4ce274f466396dcf7e3231, ASSIGN in 326 msec 2023-11-27 14:56:30,502 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:56:30,502 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701096990502"}]},"ts":"1701096990502"} 2023-11-27 14:56:30,503 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-11-27 14:56:30,505 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:56:30,512 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1590 sec 2023-11-27 14:56:31,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-27 14:56:31,453 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-11-27 14:56:31,472 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:31,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:31,474 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:31,474 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:31,474 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:31,474 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:31,475 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:31,475 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:31,476 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:31,476 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:31,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:31,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:31,476 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:31,476 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:31,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:31,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:31,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:31,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:31,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:31,482 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:31,483 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:31,483 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:31,483 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:31,484 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-27 14:56:31,907 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511305695b5ee31d04157911818527fe9ee8f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511305695b5ee31d04157911818527fe9ee8f 2023-11-27 14:56:32,312 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/35c3424dae7543e0ad40dd0f9725803d 2023-11-27 14:56:32,317 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/35c3424dae7543e0ad40dd0f9725803d as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/35c3424dae7543e0ad40dd0f9725803d 2023-11-27 14:56:32,320 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/35c3424dae7543e0ad40dd0f9725803d, entries=1000, sequenceid=5, filesize=166.0 K 2023-11-27 14:56:32,321 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 838ms, sequenceid=5, compaction requested=false 2023-11-27 14:56:32,321 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:32,322 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:32,322 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:32,322 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:32,322 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:32,322 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,322 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:32,322 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:32,327 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,327 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:32,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:32,328 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:32,328 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:32,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:32,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:32,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:32,329 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54f9d65e[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:32,329 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:32,330 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,331 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,332 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:32,332 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:32,332 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:32,332 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,332 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:32,334 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:32,334 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:32,334 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:32,336 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:32,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:32,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:32,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:32,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:32,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:32,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:32,339 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:32,339 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:32,339 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:32,339 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,339 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:32,341 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:32,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:32,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:32,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:32,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:32,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:32,342 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:32,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,344 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:32,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,351 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,351 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:32,351 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:32,351 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:32,351 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,351 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:32,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:32,351 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:32,352 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:32,351 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,352 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,352 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:32,352 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:32,352 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,352 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:32,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:32,452 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:32,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:32,457 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:32,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:32,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,458 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:32,458 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:32,458 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:32,458 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:32,459 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,459 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,460 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,460 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:32,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:32,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:32,460 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,461 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:32,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,461 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:32,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,462 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,464 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:56:32,464 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,464 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:32,464 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:32,464 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:32,465 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:32,465 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:32,465 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:32,465 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:32,465 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-27 14:56:32,881 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128dc9cd20fe21743278777134a5af745be to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128dc9cd20fe21743278777134a5af745be 2023-11-27 14:56:32,886 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/a24663dfbaec42a1929475765f22669f 2023-11-27 14:56:32,890 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/a24663dfbaec42a1929475765f22669f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a24663dfbaec42a1929475765f22669f 2023-11-27 14:56:32,894 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a24663dfbaec42a1929475765f22669f, entries=1, sequenceid=9, filesize=5.0 K 2023-11-27 14:56:32,895 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 430ms, sequenceid=9, compaction requested=false 2023-11-27 14:56:32,895 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:32,895 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:32,895 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:32,898 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,898 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:32,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:32,899 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,899 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:32,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:32,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:32,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:32,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:32,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@152c73a3[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:32,900 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:32,901 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,902 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,902 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:32,902 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:32,902 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:32,905 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,905 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:32,906 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:32,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:32,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:32,907 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:32,907 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:32,907 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:32,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:32,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:32,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:32,911 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:32,911 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:32,911 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:32,911 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:32,911 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,911 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:32,914 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,914 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:32,914 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:32,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:32,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:32,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:32,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:32,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,915 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:32,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,917 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:32,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:32,921 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:32,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:56:32,921 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:32,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:32,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:32,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:32,921 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,921 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:32,922 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:32,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:32,922 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:32,922 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:32,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:32,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:33,022 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:33,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:33,027 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:33,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:33,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,029 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:33,029 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:33,029 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:33,029 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:33,030 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,030 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,031 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:33,031 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:33,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:33,032 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,032 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:33,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:33,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:56:33,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,033 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:33,033 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:33,033 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:33,033 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:33,033 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:33,033 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:33,033 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:33,033 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:56:33,045 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201fac01212b9ef489bbd52661eb4a2e364 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201fac01212b9ef489bbd52661eb4a2e364 2023-11-27 14:56:33,049 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/c7d51d74a0be45be88036b3f873a3c6f 2023-11-27 14:56:33,053 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/c7d51d74a0be45be88036b3f873a3c6f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/c7d51d74a0be45be88036b3f873a3c6f 2023-11-27 14:56:33,056 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/c7d51d74a0be45be88036b3f873a3c6f, entries=1, sequenceid=13, filesize=4.9 K 2023-11-27 14:56:33,057 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 24ms, sequenceid=13, compaction requested=true 2023-11-27 14:56:33,057 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:33,057 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:33,057 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:33,057 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:33,057 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,057 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:33,057 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:33,060 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:33,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:33,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:33,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:33,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@762eeee1[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:33,064 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:33,064 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,066 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,066 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:33,068 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,069 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(163): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,069 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,069 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:33,069 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:33,069 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:33,069 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:33,071 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:33,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:33,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:33,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:33,072 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:33,072 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:33,072 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:33,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:33,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:33,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:33,075 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:33,075 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:33,075 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:33,075 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,075 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:33,076 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,076 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:33,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,077 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:33,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:33,077 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:33,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,079 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:33,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,084 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,084 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,084 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:33,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:33,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:56:33,085 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:33,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:33,084 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:33,084 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:33,084 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,085 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:33,085 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,085 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,085 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,085 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,085 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:33,085 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:33,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:33,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:33,185 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:33,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:33,190 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:33,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:33,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,192 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:33,192 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:33,192 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:33,192 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:33,193 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,193 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,195 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,195 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:33,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:33,195 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,196 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:33,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:33,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:56:33,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,197 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:33,197 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:33,197 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:33,197 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:33,197 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:33,197 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:33,197 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:33,197 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:56:33,608 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d6a5b2fe1a6248159d1bd51419568444 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205d6a5b2fe1a6248159d1bd51419568444 2023-11-27 14:56:33,613 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/d96afcc8abe4493ea39ca1cf6c3db5b1 2023-11-27 14:56:33,617 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/d96afcc8abe4493ea39ca1cf6c3db5b1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/d96afcc8abe4493ea39ca1cf6c3db5b1 2023-11-27 14:56:33,620 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/d96afcc8abe4493ea39ca1cf6c3db5b1, entries=1, sequenceid=17, filesize=4.9 K 2023-11-27 14:56:33,621 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 424ms, sequenceid=17, compaction requested=true 2023-11-27 14:56:33,621 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:33,621 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:33,621 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:33,621 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:33,621 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,621 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:33,621 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:33,623 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,623 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:33,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:33,624 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:33,624 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:33,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:33,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:33,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:33,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61d0169[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:33,626 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:33,626 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,627 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:33,627 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,627 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:33,629 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,629 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:33,629 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:33,629 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:33,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:33,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:33,631 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:33,631 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:33,631 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:33,631 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:33,631 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,631 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:33,633 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,633 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,633 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:33,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:33,634 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:33,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,635 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:33,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,638 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:33,638 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:56:33,638 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:33,638 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:33,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:33,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:33,638 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:33,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:33,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:33,739 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:33,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:33,743 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:33,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:33,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,744 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:33,744 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:33,745 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:33,745 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:33,745 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,745 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:33,746 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:33,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:33,747 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:33,747 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:33,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:33,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,748 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:56:33,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:33,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:33,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:33,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:33,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:33,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:33,748 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:33,748 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:33,748 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:56:34,160 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512281b6f40cd7ac04bbfb3678b4681a80f95 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512281b6f40cd7ac04bbfb3678b4681a80f95 2023-11-27 14:56:34,566 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/1d951f759d044558b85aca3f1e76298b 2023-11-27 14:56:34,569 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/1d951f759d044558b85aca3f1e76298b as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/1d951f759d044558b85aca3f1e76298b 2023-11-27 14:56:34,572 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/1d951f759d044558b85aca3f1e76298b, entries=1, sequenceid=21, filesize=4.9 K 2023-11-27 14:56:34,573 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 825ms, sequenceid=21, compaction requested=true 2023-11-27 14:56:34,573 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:34,573 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:34,573 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:34,573 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:34,573 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,573 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:34,573 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:34,575 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,575 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:34,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:34,575 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,575 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:34,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:34,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:34,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:34,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:34,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c4c6cd2[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:34,577 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:34,577 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,581 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,581 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:34,581 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:34,581 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:34,581 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,581 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:34,582 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,582 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:34,582 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:34,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:34,582 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:34,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:34,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:34,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:34,585 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:34,585 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:34,585 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:34,585 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,585 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:34,586 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,586 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:34,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:34,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,586 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:34,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:34,587 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:34,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,588 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:34,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,590 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,590 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:34,590 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:34,590 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:34,590 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:34,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:34,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:56:34,590 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:34,590 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,590 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:34,591 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,591 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:34,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:34,591 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:34,591 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:34,591 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,591 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,691 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:34,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:34,695 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:34,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:34,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,697 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:34,697 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:34,697 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:34,697 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:34,698 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,698 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,700 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:34,700 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:34,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:34,700 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,700 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:34,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:34,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,701 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:56:34,701 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,701 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:34,701 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:34,702 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:34,702 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:34,702 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:34,702 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:34,702 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:34,702 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:56:34,715 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ca10dbdcbf1c4a6a9608b8ff032e3431 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231ca10dbdcbf1c4a6a9608b8ff032e3431 2023-11-27 14:56:34,719 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/f9d61d1ed30a4df6843cb35115cae307 2023-11-27 14:56:34,722 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/f9d61d1ed30a4df6843cb35115cae307 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/f9d61d1ed30a4df6843cb35115cae307 2023-11-27 14:56:34,725 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/f9d61d1ed30a4df6843cb35115cae307, entries=1, sequenceid=25, filesize=4.9 K 2023-11-27 14:56:34,726 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 24ms, sequenceid=25, compaction requested=true 2023-11-27 14:56:34,726 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:34,726 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:34,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:34,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:34,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:34,726 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:34,728 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,728 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:34,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:34,728 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,728 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:34,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:34,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:34,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:34,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:34,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a0bb593[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:34,730 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:34,730 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,731 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,731 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:34,731 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:34,731 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:34,731 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,731 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:34,732 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:34,732 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:34,732 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,732 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:34,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:34,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:34,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:34,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:34,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:34,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:34,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:34,735 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:34,735 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:34,735 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:34,735 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,735 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:34,741 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,741 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:34,741 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:34,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:34,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:34,742 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:34,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:34,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,743 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:34,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,746 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:34,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,746 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:34,746 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:34,746 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:34,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:34,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:34,846 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:34,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:34,850 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:34,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:34,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,852 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:34,852 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:34,852 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:34,852 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:34,852 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,852 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,854 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,854 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:34,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:34,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:34,854 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,854 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:34,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,855 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:34,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,855 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,855 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:56:34,855 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,855 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:34,855 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:34,855 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:34,856 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:34,856 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:34,856 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:34,856 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:34,856 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:56:34,869 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010127acd01b131a48e186ab9db58ec93902 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010127acd01b131a48e186ab9db58ec93902 2023-11-27 14:56:34,873 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/b8c439f3609e441ea7d5fe7c8df85887 2023-11-27 14:56:34,876 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/b8c439f3609e441ea7d5fe7c8df85887 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/b8c439f3609e441ea7d5fe7c8df85887 2023-11-27 14:56:34,880 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/b8c439f3609e441ea7d5fe7c8df85887, entries=1, sequenceid=29, filesize=4.9 K 2023-11-27 14:56:34,880 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 24ms, sequenceid=29, compaction requested=true 2023-11-27 14:56:34,880 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:34,880 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:34,880 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:34,880 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:34,880 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,881 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:34,881 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:34,882 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,882 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:34,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:34,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:34,882 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,883 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:34,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:34,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:34,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:34,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b5317d5[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:34,884 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:34,884 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,886 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,886 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:34,886 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:34,886 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:34,886 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,886 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:34,887 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,887 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:34,887 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:34,887 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:34,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:34,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:34,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:34,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:34,889 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:34,889 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:34,889 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:34,889 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,889 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:34,891 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,891 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:34,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:34,891 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:34,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:34,891 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:34,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:34,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:34,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,893 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:34,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:34,895 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:34,895 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:34,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:34,895 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:34,895 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:34,896 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:34,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:34,996 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:34,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:35,000 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:35,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:35,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,002 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:35,002 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:35,002 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:35,002 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:35,002 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,002 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,004 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,004 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:35,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:35,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:35,004 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,004 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:35,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:35,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,005 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,005 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:56:35,005 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,005 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:35,005 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:35,006 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:35,006 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:35,006 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:35,006 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:35,006 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:35,006 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:56:35,416 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032a8cea57d7e043d1a27bb04fc1f1ddee to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601032a8cea57d7e043d1a27bb04fc1f1ddee 2023-11-27 14:56:35,821 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/5d3cc0af9bfa467789eea172ac9d378a 2023-11-27 14:56:35,827 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/5d3cc0af9bfa467789eea172ac9d378a as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/5d3cc0af9bfa467789eea172ac9d378a 2023-11-27 14:56:35,831 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/5d3cc0af9bfa467789eea172ac9d378a, entries=1, sequenceid=33, filesize=4.9 K 2023-11-27 14:56:35,831 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 825ms, sequenceid=33, compaction requested=true 2023-11-27 14:56:35,831 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:35,831 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:35,831 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:35,832 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:35,832 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,832 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:35,832 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:35,833 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,834 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:35,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:35,834 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,834 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:35,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:35,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:35,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:35,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:35,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ab559f[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:35,836 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:35,836 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,837 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,837 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:35,838 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:35,838 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:35,838 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,838 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:35,839 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:35,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:35,839 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:35,839 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:35,839 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:35,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:35,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:35,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:35,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:35,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:35,843 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:35,843 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:35,843 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:35,843 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,843 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:35,846 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,846 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:35,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,846 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:35,846 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:35,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:35,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,848 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:35,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,850 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,850 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:35,850 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:35,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:35,850 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:35,850 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:35,850 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,851 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:35,851 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:35,851 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,851 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,851 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,851 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:35,851 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:35,851 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:35,951 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:35,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:35,955 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:35,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:35,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,957 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:35,957 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:35,957 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:35,957 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:35,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,958 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,959 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:35,959 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:35,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:35,959 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:35,959 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:35,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:35,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,960 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,960 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:56:35,960 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:35,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:35,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:35,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:35,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:35,961 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:35,961 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:35,961 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:35,961 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:56:36,022 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:56:36,378 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509073545da20bdfb4255b787dc17c9eeaf50 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201509073545da20bdfb4255b787dc17c9eeaf50 2023-11-27 14:56:36,783 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/03170272a7c6481c96260336dae1df15 2023-11-27 14:56:36,787 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/03170272a7c6481c96260336dae1df15 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/03170272a7c6481c96260336dae1df15 2023-11-27 14:56:36,790 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/03170272a7c6481c96260336dae1df15, entries=1, sequenceid=37, filesize=4.9 K 2023-11-27 14:56:36,790 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 829ms, sequenceid=37, compaction requested=true 2023-11-27 14:56:36,790 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:36,790 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:36,790 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:36,797 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,797 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:36,797 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,797 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:36,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:36,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7afe8304[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:36,799 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:36,799 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,800 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,801 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:36,801 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:36,801 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:36,801 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,801 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:36,802 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,802 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:36,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:36,802 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:36,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:36,802 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:36,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:36,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:36,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:36,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:36,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:36,805 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:36,805 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:36,805 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:36,805 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,805 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:36,807 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,807 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,807 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:36,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:36,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:36,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:36,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:36,808 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:36,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:36,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,809 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:36,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,812 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:36,812 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,812 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:36,812 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:36,812 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:56:36,812 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,812 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:36,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:36,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:36,812 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:36,812 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:36,812 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,812 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:36,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:36,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:36,812 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,813 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,813 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,813 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,913 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:36,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:36,917 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:36,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:36,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,918 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:36,919 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:36,919 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-27 14:56:36,919 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-27 14:56:36,919 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,919 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,920 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:36,920 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:36,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:36,921 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:36,921 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:36,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:36,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-27 14:56:36,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:36,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-27 14:56:36,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:36,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-27 14:56:36,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-27 14:56:36,922 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:36,922 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:36,922 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. started... 2023-11-27 14:56:36,922 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7e7b1527d4ce274f466396dcf7e3231 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-27 14:56:37,333 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a43a366909144c54bd769b082bc12717 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120a43a366909144c54bd769b082bc12717 2023-11-27 14:56:37,738 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/a1ecbaf01a61428584edbedff898ced9 2023-11-27 14:56:37,743 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/a1ecbaf01a61428584edbedff898ced9 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a1ecbaf01a61428584edbedff898ced9 2023-11-27 14:56:37,747 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a1ecbaf01a61428584edbedff898ced9, entries=1, sequenceid=41, filesize=4.9 K 2023-11-27 14:56:37,747 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7e7b1527d4ce274f466396dcf7e3231 in 825ms, sequenceid=41, compaction requested=true 2023-11-27 14:56:37,747 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:37,747 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:37,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-27 14:56:37,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-27 14:56:37,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-27 14:56:37,748 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-27 14:56:37,750 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,750 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:37,750 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,750 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:37,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-27 14:56:37,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4dee06e5[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:37,752 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-27 14:56:37,752 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,753 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,753 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-27 14:56:37,753 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-27 14:56:37,753 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:37,755 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,755 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:37,756 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,756 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-27 14:56:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,756 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:37,756 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-27 14:56:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:37,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:37,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-27 14:56:37,758 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-27 14:56:37,759 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:37,759 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:37,759 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,759 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:37,760 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,760 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:37,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:37,760 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:37,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:37,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:37,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:37,761 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:37,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,762 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:37,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,764 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,765 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:37,765 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,765 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,765 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-27 14:56:37,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-27 14:56:37,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:37,765 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:37,765 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:37,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:37,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:37,765 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:37,765 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:37,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:37,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:37,765 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:37,865 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:37,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:37,867 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-11-27 14:56:37,868 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:56:37,869 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:56:37,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:56:37,870 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097597870 2023-11-27 14:56:37,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-27 14:56:37,871 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-11-27 14:56:37,871 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:56:37,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x3d5456f5 to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:56:37,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@480e346d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:56:37,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:56:38,701 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ba0415e10f0747cdb6d1d5a07b011985 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103ba0415e10f0747cdb6d1d5a07b011985 2023-11-27 14:56:38,703 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:56:38,720 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:56:38,722 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46354, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:56:38,727 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:56:38,728 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58782, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:56:38,736 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/4ebfaf825cf9474db26c8485d119d42a first=Optional[r5] last=Optional[r6] 2023-11-27 14:56:38,736 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/4ebfaf825cf9474db26c8485d119d42a}] 2023-11-27 14:56:38,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/4ebfaf825cf9474db26c8485d119d42a for inclusion in d7e7b1527d4ce274f466396dcf7e3231/family1 2023-11-27 14:56:38,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-11-27 14:56:38,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:56:38,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:38,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/4ebfaf825cf9474db26c8485d119d42a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ot8qkh8erl4g1v97l95g7cq1k6gc7j5d6krfmmnnvmgtb3tblmt8igpt6q6m3mnk/family1/4ebfaf825cf9474db26c8485d119d42a 2023-11-27 14:56:38,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ot8qkh8erl4g1v97l95g7cq1k6gc7j5d6krfmmnnvmgtb3tblmt8igpt6q6m3mnk/family1/4ebfaf825cf9474db26c8485d119d42a as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/079dbef56b6d40b583b5caaa27bdd954_SeqId_44_ 2023-11-27 14:56:38,744 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ot8qkh8erl4g1v97l95g7cq1k6gc7j5d6krfmmnnvmgtb3tblmt8igpt6q6m3mnk/family1/4ebfaf825cf9474db26c8485d119d42a into d7e7b1527d4ce274f466396dcf7e3231/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/079dbef56b6d40b583b5caaa27bdd954_SeqId_44_ - updating store file list. 2023-11-27 14:56:38,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/079dbef56b6d40b583b5caaa27bdd954_SeqId_44_ into d7e7b1527d4ce274f466396dcf7e3231/family1 2023-11-27 14:56:38,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ot8qkh8erl4g1v97l95g7cq1k6gc7j5d6krfmmnnvmgtb3tblmt8igpt6q6m3mnk/family1/4ebfaf825cf9474db26c8485d119d42a into d7e7b1527d4ce274f466396dcf7e3231/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/079dbef56b6d40b583b5caaa27bdd954_SeqId_44_) 2023-11-27 14:56:38,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ot8qkh8erl4g1v97l95g7cq1k6gc7j5d6krfmmnnvmgtb3tblmt8igpt6q6m3mnk/family1/4ebfaf825cf9474db26c8485d119d42a 2023-11-27 14:56:38,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-27 14:56:38,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-27 14:56:38,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): d7e7b1527d4ce274f466396dcf7e3231/family1 is initiating minor compaction 2023-11-27 14:56:38,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2dfe1428; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:56:38,752 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d7e7b1527d4ce274f466396dcf7e3231/family1 in testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:38,752 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a24663dfbaec42a1929475765f22669f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/c7d51d74a0be45be88036b3f873a3c6f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/d96afcc8abe4493ea39ca1cf6c3db5b1, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/1d951f759d044558b85aca3f1e76298b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/f9d61d1ed30a4df6843cb35115cae307, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/b8c439f3609e441ea7d5fe7c8df85887, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/5d3cc0af9bfa467789eea172ac9d378a, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/03170272a7c6481c96260336dae1df15, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a1ecbaf01a61428584edbedff898ced9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/079dbef56b6d40b583b5caaa27bdd954_SeqId_44_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp, totalSize=49.7 K 2023-11-27 14:56:38,752 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting a24663dfbaec42a1929475765f22669f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-27 14:56:38,752 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting c7d51d74a0be45be88036b3f873a3c6f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-27 14:56:38,752 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting d96afcc8abe4493ea39ca1cf6c3db5b1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-27 14:56:38,753 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 1d951f759d044558b85aca3f1e76298b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-27 14:56:38,753 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting f9d61d1ed30a4df6843cb35115cae307, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-27 14:56:38,753 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting b8c439f3609e441ea7d5fe7c8df85887, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-27 14:56:38,753 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 5d3cc0af9bfa467789eea172ac9d378a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-27 14:56:38,753 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 03170272a7c6481c96260336dae1df15, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-27 14:56:38,753 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting a1ecbaf01a61428584edbedff898ced9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-27 14:56:38,753 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 079dbef56b6d40b583b5caaa27bdd954_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-27 14:56:38,754 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:56:38,755 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:38,757 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010127acd01b131a48e186ab9db58ec93902 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010127acd01b131a48e186ab9db58ec93902 2023-11-27 14:56:38,758 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601032a8cea57d7e043d1a27bb04fc1f1ddee to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601032a8cea57d7e043d1a27bb04fc1f1ddee 2023-11-27 14:56:38,758 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-11-27 14:56:38,758 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-11-27 14:56:38,758 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:56:38,758 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:56:38,784 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): d7e7b1527d4ce274f466396dcf7e3231#family1#compaction#164 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-11-27 14:56:39,199 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ab471d3120f04c61999eb2358fdb040a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231ab471d3120f04c61999eb2358fdb040a 2023-11-27 14:56:39,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:56:39,222 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/5dfbae273e0e4f5193def03516538bb7 first=Optional[r1] last=Optional[r4] 2023-11-27 14:56:39,223 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/5dfbae273e0e4f5193def03516538bb7}] 2023-11-27 14:56:39,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/5dfbae273e0e4f5193def03516538bb7 for inclusion in d7e7b1527d4ce274f466396dcf7e3231/family1 2023-11-27 14:56:39,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-11-27 14:56:39,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:56:39,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:39,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/5dfbae273e0e4f5193def03516538bb7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__aqfvctac3cdpb72nc9nvat9mjec9ver5gb7ii0ghkrbpn6ctb1v28v53qdn80uc6/family1/5dfbae273e0e4f5193def03516538bb7 2023-11-27 14:56:39,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__aqfvctac3cdpb72nc9nvat9mjec9ver5gb7ii0ghkrbpn6ctb1v28v53qdn80uc6/family1/5dfbae273e0e4f5193def03516538bb7 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/ac690a4dde3c497e82c19003d6f9f99c_SeqId_46_ 2023-11-27 14:56:39,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__aqfvctac3cdpb72nc9nvat9mjec9ver5gb7ii0ghkrbpn6ctb1v28v53qdn80uc6/family1/5dfbae273e0e4f5193def03516538bb7 into d7e7b1527d4ce274f466396dcf7e3231/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/ac690a4dde3c497e82c19003d6f9f99c_SeqId_46_ - updating store file list. 2023-11-27 14:56:39,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/ac690a4dde3c497e82c19003d6f9f99c_SeqId_46_ into d7e7b1527d4ce274f466396dcf7e3231/family1 2023-11-27 14:56:39,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__aqfvctac3cdpb72nc9nvat9mjec9ver5gb7ii0ghkrbpn6ctb1v28v53qdn80uc6/family1/5dfbae273e0e4f5193def03516538bb7 into d7e7b1527d4ce274f466396dcf7e3231/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/ac690a4dde3c497e82c19003d6f9f99c_SeqId_46_) 2023-11-27 14:56:39,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__aqfvctac3cdpb72nc9nvat9mjec9ver5gb7ii0ghkrbpn6ctb1v28v53qdn80uc6/family1/5dfbae273e0e4f5193def03516538bb7 2023-11-27 14:56:39,235 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:56:39,236 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:39,237 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201fac01212b9ef489bbd52661eb4a2e364 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201fac01212b9ef489bbd52661eb4a2e364 2023-11-27 14:56:39,238 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205d6a5b2fe1a6248159d1bd51419568444 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205d6a5b2fe1a6248159d1bd51419568444 2023-11-27 14:56:39,238 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512281b6f40cd7ac04bbfb3678b4681a80f95 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512281b6f40cd7ac04bbfb3678b4681a80f95 2023-11-27 14:56:39,239 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231ca10dbdcbf1c4a6a9608b8ff032e3431 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231ca10dbdcbf1c4a6a9608b8ff032e3431 2023-11-27 14:56:39,239 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-11-27 14:56:39,239 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-11-27 14:56:39,240 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:56:39,240 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:56:39,596 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/fd50dbf6d0124ba7a2de22734772ec35 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/fd50dbf6d0124ba7a2de22734772ec35 2023-11-27 14:56:39,600 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d7e7b1527d4ce274f466396dcf7e3231/family1 of d7e7b1527d4ce274f466396dcf7e3231 into fd50dbf6d0124ba7a2de22734772ec35(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:39,600 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:39,600 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1701096998752; duration=0sec 2023-11-27 14:56:39,600 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:56:39,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a24663dfbaec42a1929475765f22669f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/c7d51d74a0be45be88036b3f873a3c6f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/d96afcc8abe4493ea39ca1cf6c3db5b1, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/1d951f759d044558b85aca3f1e76298b, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/f9d61d1ed30a4df6843cb35115cae307, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/b8c439f3609e441ea7d5fe7c8df85887, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/5d3cc0af9bfa467789eea172ac9d378a, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/03170272a7c6481c96260336dae1df15, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a1ecbaf01a61428584edbedff898ced9, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/079dbef56b6d40b583b5caaa27bdd954_SeqId_44_] to archive 2023-11-27 14:56:39,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:39,655 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a24663dfbaec42a1929475765f22669f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a24663dfbaec42a1929475765f22669f 2023-11-27 14:56:39,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/c7d51d74a0be45be88036b3f873a3c6f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/c7d51d74a0be45be88036b3f873a3c6f 2023-11-27 14:56:39,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/d96afcc8abe4493ea39ca1cf6c3db5b1 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/d96afcc8abe4493ea39ca1cf6c3db5b1 2023-11-27 14:56:39,658 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/1d951f759d044558b85aca3f1e76298b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/1d951f759d044558b85aca3f1e76298b 2023-11-27 14:56:39,659 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/f9d61d1ed30a4df6843cb35115cae307 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/f9d61d1ed30a4df6843cb35115cae307 2023-11-27 14:56:39,660 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/b8c439f3609e441ea7d5fe7c8df85887 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/b8c439f3609e441ea7d5fe7c8df85887 2023-11-27 14:56:39,660 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/5d3cc0af9bfa467789eea172ac9d378a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/5d3cc0af9bfa467789eea172ac9d378a 2023-11-27 14:56:39,661 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/03170272a7c6481c96260336dae1df15 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/03170272a7c6481c96260336dae1df15 2023-11-27 14:56:39,662 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a1ecbaf01a61428584edbedff898ced9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/a1ecbaf01a61428584edbedff898ced9 2023-11-27 14:56:39,663 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/079dbef56b6d40b583b5caaa27bdd954_SeqId_44_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/079dbef56b6d40b583b5caaa27bdd954_SeqId_44_ 2023-11-27 14:56:40,071 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130e5e3236f24f9493dbf8d0734570a565b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130e5e3236f24f9493dbf8d0734570a565b 2023-11-27 14:56:40,072 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:56:40,093 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a71db37a45bb4feba8fabc2010836c3f first=Optional[r00] last=Optional[r8] 2023-11-27 14:56:40,093 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a71db37a45bb4feba8fabc2010836c3f}] 2023-11-27 14:56:40,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a71db37a45bb4feba8fabc2010836c3f for inclusion in d7e7b1527d4ce274f466396dcf7e3231/family1 2023-11-27 14:56:40,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-11-27 14:56:40,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last= 2023-11-27 14:56:40,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:40,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a71db37a45bb4feba8fabc2010836c3f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__d0krt4kc7dmcl6enf4a1hgmsvn50qjnhb48vespk4btfivdbh6nr2e8egfk4v8rt/family1/a71db37a45bb4feba8fabc2010836c3f 2023-11-27 14:56:40,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__d0krt4kc7dmcl6enf4a1hgmsvn50qjnhb48vespk4btfivdbh6nr2e8egfk4v8rt/family1/a71db37a45bb4feba8fabc2010836c3f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/9dc7ce6f960a4c04828d6ab551765ea8_SeqId_49_ 2023-11-27 14:56:40,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__d0krt4kc7dmcl6enf4a1hgmsvn50qjnhb48vespk4btfivdbh6nr2e8egfk4v8rt/family1/a71db37a45bb4feba8fabc2010836c3f into d7e7b1527d4ce274f466396dcf7e3231/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/9dc7ce6f960a4c04828d6ab551765ea8_SeqId_49_ - updating store file list. 2023-11-27 14:56:40,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/9dc7ce6f960a4c04828d6ab551765ea8_SeqId_49_ into d7e7b1527d4ce274f466396dcf7e3231/family1 2023-11-27 14:56:40,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__d0krt4kc7dmcl6enf4a1hgmsvn50qjnhb48vespk4btfivdbh6nr2e8egfk4v8rt/family1/a71db37a45bb4feba8fabc2010836c3f into d7e7b1527d4ce274f466396dcf7e3231/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/9dc7ce6f960a4c04828d6ab551765ea8_SeqId_49_) 2023-11-27 14:56:40,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__d0krt4kc7dmcl6enf4a1hgmsvn50qjnhb48vespk4btfivdbh6nr2e8egfk4v8rt/family1/a71db37a45bb4feba8fabc2010836c3f 2023-11-27 14:56:40,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-27 14:56:40,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-27 14:56:40,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(1912): d7e7b1527d4ce274f466396dcf7e3231/family1 is initiating minor compaction (all files) 2023-11-27 14:56:40,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@390d92a1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:56:40,105 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d7e7b1527d4ce274f466396dcf7e3231/family1 in testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:40,105 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/35c3424dae7543e0ad40dd0f9725803d, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/fd50dbf6d0124ba7a2de22734772ec35, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/ac690a4dde3c497e82c19003d6f9f99c_SeqId_46_, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/9dc7ce6f960a4c04828d6ab551765ea8_SeqId_49_] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp, totalSize=344.3 K 2023-11-27 14:56:40,106 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 35c3424dae7543e0ad40dd0f9725803d, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878813266 2023-11-27 14:56:40,106 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting fd50dbf6d0124ba7a2de22734772ec35, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621213266 2023-11-27 14:56:40,107 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:56:40,107 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting ac690a4dde3c497e82c19003d6f9f99c_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965213266 2023-11-27 14:56:40,107 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] compactions.Compactor(207): Compacting 9dc7ce6f960a4c04828d6ab551765ea8_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014813266 2023-11-27 14:56:40,107 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:40,109 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120a43a366909144c54bd769b082bc12717 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120a43a366909144c54bd769b082bc12717 2023-11-27 14:56:40,110 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128dc9cd20fe21743278777134a5af745be to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128dc9cd20fe21743278777134a5af745be 2023-11-27 14:56:40,111 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511305695b5ee31d04157911818527fe9ee8f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511305695b5ee31d04157911818527fe9ee8f 2023-11-27 14:56:40,111 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-11-27 14:56:40,111 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-27 14:56:40,112 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:56:40,112 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:56:40,112 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-27 14:56:40,112 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-27 14:56:40,112 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:56:40,114 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-27 14:56:40,115 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.2460 sec 2023-11-27 14:56:40,132 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] throttle.PressureAwareThroughputController(145): d7e7b1527d4ce274f466396dcf7e3231#family1#compaction#165 average throughput is 11.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-11-27 14:56:40,348 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=266 (was 254) - Thread LEAK? -, OpenFileDescriptor=689 (was 662) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=139 (was 155), ProcessCount=168 (was 168), AvailableMemoryMB=7990 (was 8018) 2023-11-27 14:56:40,362 INFO [Listener at localhost/37439] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=266, OpenFileDescriptor=689, MaxFileDescriptor=60000, SystemLoadAverage=139, ProcessCount=168, AvailableMemoryMB=7990 2023-11-27 14:56:40,387 DEBUG [Listener at localhost/37439] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-11-27 14:56:40,469 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10={\xB2\x8A\x8C\xB2\x01(\x1D\xDB\xAFa\x00\x95W\xD9"\x10\x9E\xC9+.]t\xF6\xFC;K\xEE\xB9\x0D\x15u\x94*\x10B\x09\x0B\xEA\xDB\x1C\xEC%[4Te\x1F\xD7\x0D\x822\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-27 14:56:40,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure2.ProcedureExecutor(1028): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-11-27 14:56:40,471 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-11-27 14:56:40,472 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-11-27 14:56:40,472 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-27 14:56:40,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-27 14:56:40,474 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:40,475 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:40,475 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:40,475 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8 empty. 2023-11-27 14:56:40,475 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c empty. 2023-11-27 14:56:40,475 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373 empty. 2023-11-27 14:56:40,475 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:40,475 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:40,476 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:40,476 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-11-27 14:56:40,482 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-11-27 14:56:40,484 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => f50f095dfe928b667178fa9ffe5f7373, NAME => 'testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10={\xB2\x8A\x8C\xB2\x01(\x1D\xDB\xAFa\x00\x95W\xD9"\x10\x9E\xC9+.]t\xF6\xFC;K\xEE\xB9\x0D\x15u\x94*\x10B\x09\x0B\xEA\xDB\x1C\xEC%[4Te\x1F\xD7\x0D\x822\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:56:40,484 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => cbfe88cf0c0ecdf504904655dc8d73d8, NAME => 'testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10={\xB2\x8A\x8C\xB2\x01(\x1D\xDB\xAFa\x00\x95W\xD9"\x10\x9E\xC9+.]t\xF6\xFC;K\xEE\xB9\x0D\x15u\x94*\x10B\x09\x0B\xEA\xDB\x1C\xEC%[4Te\x1F\xD7\x0D\x822\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:56:40,484 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => d0e7e27ef1ca02c65babbede1720ba6c, NAME => 'testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10={\xB2\x8A\x8C\xB2\x01(\x1D\xDB\xAFa\x00\x95W\xD9"\x10\x9E\xC9+.]t\xF6\xFC;K\xEE\xB9\x0D\x15u\x94*\x10B\x09\x0B\xEA\xDB\x1C\xEC%[4Te\x1F\xD7\x0D\x822\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/.tmp 2023-11-27 14:56:40,491 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:40,491 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing f50f095dfe928b667178fa9ffe5f7373, disabling compactions & flushes 2023-11-27 14:56:40,491 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:40,491 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:40,491 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. after waiting 0 ms 2023-11-27 14:56:40,491 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:40,491 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:40,491 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:40,492 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing d0e7e27ef1ca02c65babbede1720ba6c, disabling compactions & flushes 2023-11-27 14:56:40,492 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:40,492 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:40,492 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:40,492 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. after waiting 0 ms 2023-11-27 14:56:40,492 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:40,492 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:40,492 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:40,553 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/.tmp/family1/38ffac18c6894865a0b3338b147666da as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/38ffac18c6894865a0b3338b147666da 2023-11-27 14:56:40,558 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d7e7b1527d4ce274f466396dcf7e3231/family1 of d7e7b1527d4ce274f466396dcf7e3231 into 38ffac18c6894865a0b3338b147666da(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:40,558 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:40,558 INFO [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1701097000105; duration=0sec 2023-11-27 14:56:40,559 DEBUG [RS:0;jenkins-hbase4:42669-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-27 14:56:40,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-27 14:56:40,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/35c3424dae7543e0ad40dd0f9725803d, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/fd50dbf6d0124ba7a2de22734772ec35, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/ac690a4dde3c497e82c19003d6f9f99c_SeqId_46_, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/9dc7ce6f960a4c04828d6ab551765ea8_SeqId_49_] to archive 2023-11-27 14:56:40,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:40,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/35c3424dae7543e0ad40dd0f9725803d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/35c3424dae7543e0ad40dd0f9725803d 2023-11-27 14:56:40,654 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/fd50dbf6d0124ba7a2de22734772ec35 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/fd50dbf6d0124ba7a2de22734772ec35 2023-11-27 14:56:40,655 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/ac690a4dde3c497e82c19003d6f9f99c_SeqId_46_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/ac690a4dde3c497e82c19003d6f9f99c_SeqId_46_ 2023-11-27 14:56:40,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/9dc7ce6f960a4c04828d6ab551765ea8_SeqId_49_ to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/family1/9dc7ce6f960a4c04828d6ab551765ea8_SeqId_49_ 2023-11-27 14:56:40,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-27 14:56:40,892 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:40,892 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing cbfe88cf0c0ecdf504904655dc8d73d8, disabling compactions & flushes 2023-11-27 14:56:40,892 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:40,893 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:40,893 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. after waiting 0 ms 2023-11-27 14:56:40,893 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:40,893 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:40,893 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:40,895 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-11-27 14:56:40,896 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701097000896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701097000896"}]},"ts":"1701097000896"} 2023-11-27 14:56:40,896 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1701097000896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701097000896"}]},"ts":"1701097000896"} 2023-11-27 14:56:40,896 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701097000896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1701097000896"}]},"ts":"1701097000896"} 2023-11-27 14:56:40,898 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-27 14:56:40,899 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-27 14:56:40,899 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701097000899"}]},"ts":"1701097000899"} 2023-11-27 14:56:40,900 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-11-27 14:56:40,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cbfe88cf0c0ecdf504904655dc8d73d8, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d0e7e27ef1ca02c65babbede1720ba6c, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f50f095dfe928b667178fa9ffe5f7373, ASSIGN}] 2023-11-27 14:56:40,905 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d0e7e27ef1ca02c65babbede1720ba6c, ASSIGN 2023-11-27 14:56:40,906 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cbfe88cf0c0ecdf504904655dc8d73d8, ASSIGN 2023-11-27 14:56:40,906 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f50f095dfe928b667178fa9ffe5f7373, ASSIGN 2023-11-27 14:56:40,906 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cbfe88cf0c0ecdf504904655dc8d73d8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:56:40,906 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d0e7e27ef1ca02c65babbede1720ba6c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:56:40,907 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f50f095dfe928b667178fa9ffe5f7373, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42669,1701096850360; forceNewPlan=false, retain=false 2023-11-27 14:56:41,058 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=f50f095dfe928b667178fa9ffe5f7373, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:41,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=d0e7e27ef1ca02c65babbede1720ba6c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:41,058 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=cbfe88cf0c0ecdf504904655dc8d73d8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:41,058 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1701097001058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701097001058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701097001058"}]},"ts":"1701097001058"} 2023-11-27 14:56:41,058 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701097001058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701097001058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701097001058"}]},"ts":"1701097001058"} 2023-11-27 14:56:41,058 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701097001058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1701097001058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1701097001058"}]},"ts":"1701097001058"} 2023-11-27 14:56:41,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=120, ppid=118, state=RUNNABLE; OpenRegionProcedure d0e7e27ef1ca02c65babbede1720ba6c, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:41,060 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=121, ppid=117, state=RUNNABLE; OpenRegionProcedure cbfe88cf0c0ecdf504904655dc8d73d8, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:41,063 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure f50f095dfe928b667178fa9ffe5f7373, server=jenkins-hbase4.apache.org,42669,1701096850360}] 2023-11-27 14:56:41,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-27 14:56:41,212 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:41,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:41,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cbfe88cf0c0ecdf504904655dc8d73d8, NAME => 'testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8.', STARTKEY => '', ENDKEY => '1'} 2023-11-27 14:56:41,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:41,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:41,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:41,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:41,238 INFO [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:41,240 DEBUG [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1 2023-11-27 14:56:41,240 DEBUG [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1 2023-11-27 14:56:41,240 INFO [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cbfe88cf0c0ecdf504904655dc8d73d8 columnFamilyName family1 2023-11-27 14:56:41,242 INFO [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] regionserver.HStore(310): Store=cbfe88cf0c0ecdf504904655dc8d73d8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:41,242 INFO [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:41,243 DEBUG [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2 2023-11-27 14:56:41,243 DEBUG [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2 2023-11-27 14:56:41,244 INFO [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cbfe88cf0c0ecdf504904655dc8d73d8 columnFamilyName family2 2023-11-27 14:56:41,245 INFO [StoreOpener-cbfe88cf0c0ecdf504904655dc8d73d8-1] regionserver.HStore(310): Store=cbfe88cf0c0ecdf504904655dc8d73d8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:41,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:41,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:41,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:56:41,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:41,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:56:41,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cbfe88cf0c0ecdf504904655dc8d73d8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61070221, jitterRate=-0.08998279273509979}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:56:41,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:41,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8., pid=121, masterSystemTime=1701097001212 2023-11-27 14:56:41,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:41,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:41,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:41,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d0e7e27ef1ca02c65babbede1720ba6c, NAME => 'testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c.', STARTKEY => '1', ENDKEY => '2'} 2023-11-27 14:56:41,254 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=cbfe88cf0c0ecdf504904655dc8d73d8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:41,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:41,254 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701097001254"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701097001254"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701097001254"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701097001254"}]},"ts":"1701097001254"} 2023-11-27 14:56:41,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:41,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:41,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:41,256 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=121, resume processing ppid=117 2023-11-27 14:56:41,256 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=121, ppid=117, state=SUCCESS; OpenRegionProcedure cbfe88cf0c0ecdf504904655dc8d73d8, server=jenkins-hbase4.apache.org,42669,1701096850360 in 195 msec 2023-11-27 14:56:41,257 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cbfe88cf0c0ecdf504904655dc8d73d8, ASSIGN in 352 msec 2023-11-27 14:56:41,274 INFO [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:41,276 DEBUG [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1 2023-11-27 14:56:41,276 DEBUG [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1 2023-11-27 14:56:41,276 INFO [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0e7e27ef1ca02c65babbede1720ba6c columnFamilyName family1 2023-11-27 14:56:41,276 INFO [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] regionserver.HStore(310): Store=d0e7e27ef1ca02c65babbede1720ba6c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:41,277 INFO [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:41,277 DEBUG [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2 2023-11-27 14:56:41,277 DEBUG [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2 2023-11-27 14:56:41,278 INFO [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0e7e27ef1ca02c65babbede1720ba6c columnFamilyName family2 2023-11-27 14:56:41,278 INFO [StoreOpener-d0e7e27ef1ca02c65babbede1720ba6c-1] regionserver.HStore(310): Store=d0e7e27ef1ca02c65babbede1720ba6c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:41,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:41,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:41,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:56:41,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:41,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:56:41,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d0e7e27ef1ca02c65babbede1720ba6c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67954640, jitterRate=0.012603044509887695}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:56:41,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:41,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c., pid=120, masterSystemTime=1701097001212 2023-11-27 14:56:41,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:41,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:41,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:41,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f50f095dfe928b667178fa9ffe5f7373, NAME => 'testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373.', STARTKEY => '2', ENDKEY => ''} 2023-11-27 14:56:41,284 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=d0e7e27ef1ca02c65babbede1720ba6c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:41,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:41,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-27 14:56:41,284 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1701097001284"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701097001284"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701097001284"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701097001284"}]},"ts":"1701097001284"} 2023-11-27 14:56:41,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:41,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:41,286 INFO [PEWorker-3] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=120, resume processing ppid=118 2023-11-27 14:56:41,287 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=120, ppid=118, state=SUCCESS; OpenRegionProcedure d0e7e27ef1ca02c65babbede1720ba6c, server=jenkins-hbase4.apache.org,42669,1701096850360 in 225 msec 2023-11-27 14:56:41,287 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d0e7e27ef1ca02c65babbede1720ba6c, ASSIGN in 383 msec 2023-11-27 14:56:41,307 INFO [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:41,308 DEBUG [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1 2023-11-27 14:56:41,308 DEBUG [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1 2023-11-27 14:56:41,308 INFO [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f50f095dfe928b667178fa9ffe5f7373 columnFamilyName family1 2023-11-27 14:56:41,309 INFO [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] regionserver.HStore(310): Store=f50f095dfe928b667178fa9ffe5f7373/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:41,309 INFO [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:41,310 DEBUG [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2 2023-11-27 14:56:41,310 DEBUG [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2 2023-11-27 14:56:41,310 INFO [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f50f095dfe928b667178fa9ffe5f7373 columnFamilyName family2 2023-11-27 14:56:41,310 INFO [StoreOpener-f50f095dfe928b667178fa9ffe5f7373-1] regionserver.HStore(310): Store=f50f095dfe928b667178fa9ffe5f7373/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-27 14:56:41,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:41,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:41,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-27 14:56:41,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:41,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-27 14:56:41,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f50f095dfe928b667178fa9ffe5f7373; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66814369, jitterRate=-0.0043883174657821655}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-27 14:56:41,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:41,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373., pid=122, masterSystemTime=1701097001212 2023-11-27 14:56:41,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:41,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:41,317 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=f50f095dfe928b667178fa9ffe5f7373, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:41,317 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1701097001317"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1701097001317"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1701097001317"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1701097001317"}]},"ts":"1701097001317"} 2023-11-27 14:56:41,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=122, resume processing ppid=119 2023-11-27 14:56:41,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure f50f095dfe928b667178fa9ffe5f7373, server=jenkins-hbase4.apache.org,42669,1701096850360 in 257 msec 2023-11-27 14:56:41,321 INFO [PEWorker-5] procedure2.ProcedureExecutor(1825): Finished subprocedure pid=119, resume processing ppid=116 2023-11-27 14:56:41,321 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f50f095dfe928b667178fa9ffe5f7373, ASSIGN in 415 msec 2023-11-27 14:56:41,321 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-27 14:56:41,321 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1701097001321"}]},"ts":"1701097001321"} 2023-11-27 14:56:41,322 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-11-27 14:56:41,324 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-11-27 14:56:41,325 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 855 msec 2023-11-27 14:56:41,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-27 14:56:41,577 INFO [Listener at localhost/37439] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-11-27 14:56:41,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:41,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:41,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:41,590 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:41,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-27 14:56:41,597 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-27 14:56:41,597 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:41,597 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-27 14:56:41,597 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-27 14:56:41,598 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:41,598 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:41,599 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:41,599 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:41,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:41,599 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:41,600 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:41,600 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:41,600 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-27 14:56:41,604 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-27 14:56:41,604 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-27 14:56:41,604 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:41,604 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-27 14:56:41,604 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:41,604 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:41,604 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:41,604 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:41,604 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. started... 2023-11-27 14:56:41,604 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. started... 2023-11-27 14:56:41,604 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:41,604 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. started... 2023-11-27 14:56:41,606 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cbfe88cf0c0ecdf504904655dc8d73d8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-27 14:56:41,606 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d0e7e27ef1ca02c65babbede1720ba6c 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-27 14:56:41,606 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f50f095dfe928b667178fa9ffe5f7373 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-27 14:56:42,060 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127daaf643b2e1d4646a0dcbbae6d8704c8 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127daaf643b2e1d4646a0dcbbae6d8704c8 2023-11-27 14:56:42,064 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127035ecc4513c74ab3b0a0414e80cf1fd8 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127035ecc4513c74ab3b0a0414e80cf1fd8 2023-11-27 14:56:42,064 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311277fae167b78c74157b7f442397bcd5bea to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311277fae167b78c74157b7f442397bcd5bea 2023-11-27 14:56:42,066 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-11-27 14:56:42,081 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/0dcd684ad4b442aa8a4151412f73daa2 2023-11-27 14:56:42,082 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/38016a82c820476399e57e6d2a9c2a08 2023-11-27 14:56:42,481 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/a3f37411a819439ea7803f577e986f98 2023-11-27 14:56:42,513 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311274be41bcc701248b0b6cf13a1766c3204 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202311274be41bcc701248b0b6cf13a1766c3204 2023-11-27 14:56:42,514 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311272fd45a8eebd340e6ac065a179ad0b3d4 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202311272fd45a8eebd340e6ac065a179ad0b3d4 2023-11-27 14:56:42,914 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127bf30e31fd3b24e5abfa763fe4f0e3851 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231127bf30e31fd3b24e5abfa763fe4f0e3851 2023-11-27 14:56:42,918 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/0f8bce2c6e624154abdedddb63d96ead 2023-11-27 14:56:42,922 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/80bb038a61654a3a9b25be75bb232d9e 2023-11-27 14:56:42,922 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/0dcd684ad4b442aa8a4151412f73daa2 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/0dcd684ad4b442aa8a4151412f73daa2 2023-11-27 14:56:42,925 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/38016a82c820476399e57e6d2a9c2a08 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/38016a82c820476399e57e6d2a9c2a08 2023-11-27 14:56:42,945 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/0dcd684ad4b442aa8a4151412f73daa2, entries=4, sequenceid=5, filesize=5.5 K 2023-11-27 14:56:42,946 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/0f8bce2c6e624154abdedddb63d96ead as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/0f8bce2c6e624154abdedddb63d96ead 2023-11-27 14:56:42,949 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/0f8bce2c6e624154abdedddb63d96ead, entries=2, sequenceid=5, filesize=5.1 K 2023-11-27 14:56:42,950 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d0e7e27ef1ca02c65babbede1720ba6c in 1345ms, sequenceid=5, compaction requested=false 2023-11-27 14:56:42,951 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:42,951 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:42,951 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:42,959 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/38016a82c820476399e57e6d2a9c2a08, entries=4, sequenceid=5, filesize=5.5 K 2023-11-27 14:56:42,960 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/80bb038a61654a3a9b25be75bb232d9e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/80bb038a61654a3a9b25be75bb232d9e 2023-11-27 14:56:42,963 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/80bb038a61654a3a9b25be75bb232d9e, entries=2, sequenceid=5, filesize=5.1 K 2023-11-27 14:56:42,964 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f50f095dfe928b667178fa9ffe5f7373 in 1358ms, sequenceid=5, compaction requested=false 2023-11-27 14:56:42,964 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:42,964 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:42,964 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:43,319 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/3e2cca46713743718603924a0b905184 2023-11-27 14:56:43,324 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/a3f37411a819439ea7803f577e986f98 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/a3f37411a819439ea7803f577e986f98 2023-11-27 14:56:43,352 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/a3f37411a819439ea7803f577e986f98, entries=4, sequenceid=5, filesize=5.5 K 2023-11-27 14:56:43,353 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/3e2cca46713743718603924a0b905184 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3e2cca46713743718603924a0b905184 2023-11-27 14:56:43,356 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3e2cca46713743718603924a0b905184, entries=2, sequenceid=5, filesize=5.1 K 2023-11-27 14:56:43,356 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for cbfe88cf0c0ecdf504904655dc8d73d8 in 1751ms, sequenceid=5, compaction requested=false 2023-11-27 14:56:43,356 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:43,356 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:43,357 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:43,357 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:43,357 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,357 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-27 14:56:43,357 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-27 14:56:43,358 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:43,358 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:43,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:43,358 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:43,358 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:43,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-27 14:56:43,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e704d32[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:43,360 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-27 14:56:43,360 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:43,361 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:43,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:43,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:43,361 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-27 14:56:43,361 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-27 14:56:43,361 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:43,361 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,361 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:43,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-27 14:56:43,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:43,362 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,362 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-27 14:56:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:43,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:43,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:43,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-27 14:56:43,365 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-27 14:56:43,365 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:43,365 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:43,365 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-27 14:56:43,365 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:43,371 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:43,371 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:43,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:43,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:43,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:43,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:43,371 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:43,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:43,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:43,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:43,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:43,371 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:43,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:43,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:43,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:43,372 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:43,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:43,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,375 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:43,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-27 14:56:43,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-27 14:56:43,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:43,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:43,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:43,375 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:43,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:43,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:43,475 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:43,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:43,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:43,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:43,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:43,479 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:43,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:43,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-27 14:56:43,481 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-27 14:56:43,481 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:43,481 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-27 14:56:43,481 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-27 14:56:43,482 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:43,482 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:43,483 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:43,483 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:43,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:43,483 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:43,484 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:43,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:43,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:43,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:43,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:43,484 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-27 14:56:43,484 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-27 14:56:43,484 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-27 14:56:43,485 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-27 14:56:43,485 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:43,485 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-27 14:56:43,485 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:43,485 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:43,485 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:43,485 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:43,485 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. started... 2023-11-27 14:56:43,485 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:43,485 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. started... 2023-11-27 14:56:43,485 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f50f095dfe928b667178fa9ffe5f7373 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-27 14:56:43,485 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. started... 2023-11-27 14:56:43,485 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cbfe88cf0c0ecdf504904655dc8d73d8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-27 14:56:43,485 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d0e7e27ef1ca02c65babbede1720ba6c 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-27 14:56:43,527 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112703e33662836146478d1f804b6eea6c93 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023112703e33662836146478d1f804b6eea6c93 2023-11-27 14:56:43,543 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/9199780bf15341fcb4bc620d08a30c71 2023-11-27 14:56:43,927 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311270f99d1b9081749a88baad2be396a308a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311270f99d1b9081749a88baad2be396a308a 2023-11-27 14:56:43,927 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311272f08d1fada274a58af398be2e3c8809b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311272f08d1fada274a58af398be2e3c8809b 2023-11-27 14:56:43,942 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/28b67c6724324952b51fb3cb80202eb2 2023-11-27 14:56:43,975 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127c06a40078ff942cd8a5fc0500f3d8f1a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231127c06a40078ff942cd8a5fc0500f3d8f1a 2023-11-27 14:56:43,978 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/297fc5fcdd0e4efbb8478f47f6615984 2023-11-27 14:56:43,981 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/9199780bf15341fcb4bc620d08a30c71 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/9199780bf15341fcb4bc620d08a30c71 2023-11-27 14:56:44,004 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/9199780bf15341fcb4bc620d08a30c71, entries=4, sequenceid=9, filesize=5.5 K 2023-11-27 14:56:44,005 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/297fc5fcdd0e4efbb8478f47f6615984 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/297fc5fcdd0e4efbb8478f47f6615984 2023-11-27 14:56:44,008 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/297fc5fcdd0e4efbb8478f47f6615984, entries=2, sequenceid=9, filesize=5.1 K 2023-11-27 14:56:44,008 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for cbfe88cf0c0ecdf504904655dc8d73d8 in 523ms, sequenceid=9, compaction requested=false 2023-11-27 14:56:44,008 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:44,008 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:44,008 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:44,341 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/2caab41acb6041f3a2c195075a7d93a3 2023-11-27 14:56:44,371 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127c68c493286ab4e7aaa1195207c626908 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20231127c68c493286ab4e7aaa1195207c626908 2023-11-27 14:56:44,372 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023112746b3491bf8024f09addb1cd032a02dae to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023112746b3491bf8024f09addb1cd032a02dae 2023-11-27 14:56:44,375 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/9e7a5e86abde458d9e3aa8f0c17b4311 2023-11-27 14:56:44,376 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/1f89e202540a44a691c8ddcb9b88b836 2023-11-27 14:56:44,379 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/28b67c6724324952b51fb3cb80202eb2 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/28b67c6724324952b51fb3cb80202eb2 2023-11-27 14:56:44,380 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/2caab41acb6041f3a2c195075a7d93a3 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/2caab41acb6041f3a2c195075a7d93a3 2023-11-27 14:56:44,402 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/28b67c6724324952b51fb3cb80202eb2, entries=4, sequenceid=9, filesize=5.5 K 2023-11-27 14:56:44,403 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/9e7a5e86abde458d9e3aa8f0c17b4311 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/9e7a5e86abde458d9e3aa8f0c17b4311 2023-11-27 14:56:44,403 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/2caab41acb6041f3a2c195075a7d93a3, entries=4, sequenceid=9, filesize=5.5 K 2023-11-27 14:56:44,405 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/1f89e202540a44a691c8ddcb9b88b836 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/1f89e202540a44a691c8ddcb9b88b836 2023-11-27 14:56:44,407 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/9e7a5e86abde458d9e3aa8f0c17b4311, entries=2, sequenceid=9, filesize=5.1 K 2023-11-27 14:56:44,408 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f50f095dfe928b667178fa9ffe5f7373 in 923ms, sequenceid=9, compaction requested=false 2023-11-27 14:56:44,408 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:44,408 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:44,408 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:44,409 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/1f89e202540a44a691c8ddcb9b88b836, entries=2, sequenceid=9, filesize=5.1 K 2023-11-27 14:56:44,409 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d0e7e27ef1ca02c65babbede1720ba6c in 924ms, sequenceid=9, compaction requested=false 2023-11-27 14:56:44,410 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:44,410 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:44,410 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:44,410 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:44,410 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,410 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-27 14:56:44,410 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-27 14:56:44,412 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,412 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:44,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:44,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:44,413 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:44,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:44,413 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:44,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:44,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:44,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:44,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-27 14:56:44,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6128f0ce[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:44,414 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-27 14:56:44,414 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:44,416 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:44,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:44,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:44,416 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-27 14:56:44,416 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-27 14:56:44,416 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:44,416 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,416 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:44,417 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-27 14:56:44,417 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:44,417 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,417 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-27 14:56:44,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:44,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:44,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:44,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:44,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:44,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:44,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:44,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:44,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-27 14:56:44,419 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-27 14:56:44,419 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:44,420 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:44,420 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-27 14:56:44,420 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:44,421 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:44,421 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:44,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:44,421 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:44,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:44,422 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:44,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:44,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:44,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:44,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:44,423 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:44,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:44,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,425 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,425 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:44,425 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:44,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:44,425 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:44,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:44,426 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:44,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-27 14:56:44,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:44,426 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:44,426 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,426 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:44,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:44,426 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:44,426 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:44,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-27 14:56:44,426 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:44,426 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:44,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:44,526 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:44,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:44,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:44,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:44,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:44,530 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:44,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:44,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-27 14:56:44,532 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-27 14:56:44,532 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:44,532 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-27 14:56:44,532 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-27 14:56:44,533 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:44,533 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:44,534 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:44,534 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:44,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:44,534 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:44,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:44,534 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:44,535 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:44,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:44,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:44,535 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-27 14:56:44,535 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-27 14:56:44,535 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-27 14:56:44,536 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-27 14:56:44,536 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:44,536 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-27 14:56:44,536 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:44,536 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. started... 2023-11-27 14:56:44,536 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:44,536 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:44,536 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:44,536 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cbfe88cf0c0ecdf504904655dc8d73d8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-27 14:56:44,536 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:44,536 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. started... 2023-11-27 14:56:44,536 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. started... 2023-11-27 14:56:44,536 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f50f095dfe928b667178fa9ffe5f7373 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-27 14:56:44,536 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d0e7e27ef1ca02c65babbede1720ba6c 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-27 14:56:44,979 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127f7bc58594b944fe787577aca64ad6e61 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127f7bc58594b944fe787577aca64ad6e61 2023-11-27 14:56:44,979 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127360161170fb941f99c4e80f31cb0ff14 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231127360161170fb941f99c4e80f31cb0ff14 2023-11-27 14:56:44,979 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311275c5d201cff814f068d1cf190fc6d0a27 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202311275c5d201cff814f068d1cf190fc6d0a27 2023-11-27 14:56:44,994 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/62cddfe482514e56a8dd75b85b7d17be 2023-11-27 14:56:44,995 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/cc6d594b87044fec864bb91908867cac 2023-11-27 14:56:45,395 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/d044412080064134b550732722d6524f 2023-11-27 14:56:45,432 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127c24727ea3509447f9d74365b2356141c to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231127c24727ea3509447f9d74365b2356141c 2023-11-27 14:56:45,432 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023112713da97b0bd51449093c966db88e875f7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023112713da97b0bd51449093c966db88e875f7 2023-11-27 14:56:45,433 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023112730c586a5620d48d19c6c180487bf60d9 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023112730c586a5620d48d19c6c180487bf60d9 2023-11-27 14:56:45,840 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/303a8053de724ae4a6243745bc2a18ca 2023-11-27 14:56:45,844 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/d044412080064134b550732722d6524f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/d044412080064134b550732722d6524f 2023-11-27 14:56:45,848 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/1e9e46c4f4534f598dbd49f23b054f4e 2023-11-27 14:56:45,853 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/62cddfe482514e56a8dd75b85b7d17be as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/62cddfe482514e56a8dd75b85b7d17be 2023-11-27 14:56:45,853 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/f2e7c42c3925472999717f975a339156 2023-11-27 14:56:45,859 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/cc6d594b87044fec864bb91908867cac as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/cc6d594b87044fec864bb91908867cac 2023-11-27 14:56:45,872 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/d044412080064134b550732722d6524f, entries=4, sequenceid=13, filesize=5.5 K 2023-11-27 14:56:45,873 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/303a8053de724ae4a6243745bc2a18ca as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/303a8053de724ae4a6243745bc2a18ca 2023-11-27 14:56:45,877 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/303a8053de724ae4a6243745bc2a18ca, entries=2, sequenceid=13, filesize=5.1 K 2023-11-27 14:56:45,878 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d0e7e27ef1ca02c65babbede1720ba6c in 1342ms, sequenceid=13, compaction requested=true 2023-11-27 14:56:45,878 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:45,878 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:45,878 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:45,881 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/62cddfe482514e56a8dd75b85b7d17be, entries=4, sequenceid=13, filesize=5.5 K 2023-11-27 14:56:45,884 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/1e9e46c4f4534f598dbd49f23b054f4e as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/1e9e46c4f4534f598dbd49f23b054f4e 2023-11-27 14:56:45,885 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/cc6d594b87044fec864bb91908867cac, entries=4, sequenceid=13, filesize=5.5 K 2023-11-27 14:56:45,886 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/f2e7c42c3925472999717f975a339156 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/f2e7c42c3925472999717f975a339156 2023-11-27 14:56:45,902 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/f2e7c42c3925472999717f975a339156, entries=2, sequenceid=13, filesize=5.1 K 2023-11-27 14:56:45,902 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/1e9e46c4f4534f598dbd49f23b054f4e, entries=2, sequenceid=13, filesize=5.1 K 2023-11-27 14:56:45,903 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for cbfe88cf0c0ecdf504904655dc8d73d8 in 1367ms, sequenceid=13, compaction requested=true 2023-11-27 14:56:45,903 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f50f095dfe928b667178fa9ffe5f7373 in 1367ms, sequenceid=13, compaction requested=true 2023-11-27 14:56:45,903 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:45,903 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:45,903 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:45,903 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:45,903 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:45,903 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:45,903 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:45,903 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,903 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-27 14:56:45,903 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-27 14:56:45,906 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,906 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:45,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:45,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:45,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:45,906 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:45,906 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:45,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:45,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:45,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:45,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-27 14:56:45,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f40a94c[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:45,908 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-27 14:56:45,908 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:45,909 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:45,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:45,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:45,909 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-27 14:56:45,910 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-27 14:56:45,910 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:45,910 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,910 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:45,912 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-27 14:56:45,913 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,913 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:45,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,913 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-27 14:56:45,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:45,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:45,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:45,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:45,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:45,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:45,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:45,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:45,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-27 14:56:45,916 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-27 14:56:45,916 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:45,916 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:45,916 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-27 14:56:45,916 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:45,918 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:45,918 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:45,918 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:45,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:45,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:45,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:45,919 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:45,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:45,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:45,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:45,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:45,920 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:45,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:45,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,922 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:45,923 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-27 14:56:45,923 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:45,923 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:45,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:45,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:45,923 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:45,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:45,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:45,923 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:46,023 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:46,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:46,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:46,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:46,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. with WAL disabled. Data may be lost in the event of a crash. 2023-11-27 14:56:46,027 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:46,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:46,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-27 14:56:46,029 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-27 14:56:46,029 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:46,029 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-27 14:56:46,029 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-27 14:56:46,030 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:46,030 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:46,031 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:46,031 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:46,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:46,031 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:46,031 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:46,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:46,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-27 14:56:46,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-27 14:56:46,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-27 14:56:46,032 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-27 14:56:46,032 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:46,032 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-27 14:56:46,033 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:46,033 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:46,033 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:46,033 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. started... 2023-11-27 14:56:46,033 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:46,033 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:46,033 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cbfe88cf0c0ecdf504904655dc8d73d8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-27 14:56:46,033 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. started... 2023-11-27 14:56:46,033 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. started... 2023-11-27 14:56:46,033 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f50f095dfe928b667178fa9ffe5f7373 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-27 14:56:46,033 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d0e7e27ef1ca02c65babbede1720ba6c 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-27 14:56:46,098 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127643be05eb02748b6ad284bdb40fb6dd2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127643be05eb02748b6ad284bdb40fb6dd2 2023-11-27 14:56:46,236 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:56:46,236 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64b8e4ef to 127.0.0.1:61098 2023-11-27 14:56:46,236 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:46,237 INFO [Finalizer] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:56:46,237 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3d5456f5 to 127.0.0.1:61098 2023-11-27 14:56:46,237 DEBUG [Finalizer] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:46,481 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127b987fba0c754400ea5e7f7262f6be892 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231127b987fba0c754400ea5e7f7262f6be892 2023-11-27 14:56:46,482 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127a8a3893c33044c44b17cbb70bfd1c2e0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127a8a3893c33044c44b17cbb70bfd1c2e0 2023-11-27 14:56:46,521 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/b2617319713f4706be4015d1fe264512 2023-11-27 14:56:46,563 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127d13b4b80dbca4e0183355856b8c8c698 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231127d13b4b80dbca4e0183355856b8c8c698 2023-11-27 14:56:46,567 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/924a9cf301a04f2dae845bedb50681cf 2023-11-27 14:56:46,571 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/b2617319713f4706be4015d1fe264512 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/b2617319713f4706be4015d1fe264512 2023-11-27 14:56:46,600 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/b2617319713f4706be4015d1fe264512, entries=4, sequenceid=17, filesize=5.5 K 2023-11-27 14:56:46,601 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/924a9cf301a04f2dae845bedb50681cf as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/924a9cf301a04f2dae845bedb50681cf 2023-11-27 14:56:46,604 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/924a9cf301a04f2dae845bedb50681cf, entries=2, sequenceid=17, filesize=5.1 K 2023-11-27 14:56:46,605 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for cbfe88cf0c0ecdf504904655dc8d73d8 in 572ms, sequenceid=17, compaction requested=true 2023-11-27 14:56:46,605 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:46,605 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:46,605 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:46,792 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-27 14:56:46,905 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/a0ea89951e6345f8af2625bfaae5f0be 2023-11-27 14:56:46,905 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/2be0e6f3e12f4021954b97b4739ec2cf 2023-11-27 14:56:47,336 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127c37e5af6f1034b7a9ec386af8315c4e7 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20231127c37e5af6f1034b7a9ec386af8315c4e7 2023-11-27 14:56:47,338 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127556cc4310c7a445d92a3d874f264fc2d to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231127556cc4310c7a445d92a3d874f264fc2d 2023-11-27 14:56:47,341 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/a7cea3e5333246ce93fa02a35951eabf 2023-11-27 14:56:47,342 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/de8d5800d9c947b0b4205175ed7e8db6 2023-11-27 14:56:47,345 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/2be0e6f3e12f4021954b97b4739ec2cf as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/2be0e6f3e12f4021954b97b4739ec2cf 2023-11-27 14:56:47,346 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/a0ea89951e6345f8af2625bfaae5f0be as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/a0ea89951e6345f8af2625bfaae5f0be 2023-11-27 14:56:47,381 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/2be0e6f3e12f4021954b97b4739ec2cf, entries=4, sequenceid=17, filesize=5.5 K 2023-11-27 14:56:47,381 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/a0ea89951e6345f8af2625bfaae5f0be, entries=4, sequenceid=17, filesize=5.5 K 2023-11-27 14:56:47,382 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/a7cea3e5333246ce93fa02a35951eabf as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/a7cea3e5333246ce93fa02a35951eabf 2023-11-27 14:56:47,382 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/de8d5800d9c947b0b4205175ed7e8db6 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/de8d5800d9c947b0b4205175ed7e8db6 2023-11-27 14:56:47,389 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/a7cea3e5333246ce93fa02a35951eabf, entries=2, sequenceid=17, filesize=5.1 K 2023-11-27 14:56:47,389 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/de8d5800d9c947b0b4205175ed7e8db6, entries=2, sequenceid=17, filesize=5.1 K 2023-11-27 14:56:47,390 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f50f095dfe928b667178fa9ffe5f7373 in 1357ms, sequenceid=17, compaction requested=true 2023-11-27 14:56:47,390 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d0e7e27ef1ca02c65babbede1720ba6c in 1357ms, sequenceid=17, compaction requested=true 2023-11-27 14:56:47,390 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:47,390 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:47,390 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:47,390 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:47,390 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:47,390 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:47,390 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:47,390 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,390 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-27 14:56:47,390 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-27 14:56:47,392 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,392 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:47,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:47,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:47,392 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:47,392 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:47,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:47,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:47,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:47,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:47,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-27 14:56:47,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5224536f[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:47,393 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-27 14:56:47,394 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:47,399 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:47,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:47,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:47,399 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-27 14:56:47,399 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-27 14:56:47,399 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:47,399 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,399 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:47,401 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,401 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-27 14:56:47,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:47,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:47,401 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:47,401 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-27 14:56:47,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:47,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:47,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:47,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:47,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:47,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:47,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-27 14:56:47,403 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-27 14:56:47,403 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:47,403 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:47,403 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-27 14:56:47,403 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:47,405 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:47,405 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:47,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:47,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:47,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:47,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:47,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:47,405 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:47,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:47,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:47,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:47,406 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:47,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:47,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:47,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:47,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,407 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:47,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:47,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:47,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:47,409 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:47,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:47,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:47,409 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-27 14:56:47,409 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:47,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:47,409 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:47,410 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:47,410 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:47,410 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:47,410 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:47,410 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:47,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:47,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:47,510 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:47,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:47,518 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:47,525 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:47,532 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:47,533 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:47,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-27 14:56:47,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-27 14:56:47,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-27 14:56:47,618 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-27 14:56:47,618 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:47,618 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-27 14:56:47,618 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-27 14:56:47,619 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:47,619 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:47,621 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,621 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:47,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:47,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:47,621 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:47,621 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-27 14:56:47,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:47,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:47,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-27 14:56:47,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:47,622 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-27 14:56:47,622 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-27 14:56:47,622 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-27 14:56:47,622 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-27 14:56:47,622 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-27 14:56:47,622 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-27 14:56:47,622 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:47,622 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-27 14:56:47,622 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. started... 2023-11-27 14:56:47,622 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:47,622 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:47,622 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cbfe88cf0c0ecdf504904655dc8d73d8 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-27 14:56:47,622 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. started... 2023-11-27 14:56:47,622 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-27 14:56:47,623 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f50f095dfe928b667178fa9ffe5f7373 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-27 14:56:47,622 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. started... 2023-11-27 14:56:47,623 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d0e7e27ef1ca02c65babbede1720ba6c 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-27 14:56:47,665 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/792d6bdcabf848d1afbaecba533182c1 2023-11-27 14:56:47,695 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 792d6bdcabf848d1afbaecba533182c1 2023-11-27 14:56:47,724 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/7b756279af8c43f2b618ff51dcdafcdc 2023-11-27 14:56:47,727 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b756279af8c43f2b618ff51dcdafcdc 2023-11-27 14:56:47,728 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/792d6bdcabf848d1afbaecba533182c1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/792d6bdcabf848d1afbaecba533182c1 2023-11-27 14:56:47,751 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 792d6bdcabf848d1afbaecba533182c1 2023-11-27 14:56:47,751 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/792d6bdcabf848d1afbaecba533182c1, entries=3, sequenceid=23, filesize=5.2 K 2023-11-27 14:56:47,752 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/7b756279af8c43f2b618ff51dcdafcdc as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/7b756279af8c43f2b618ff51dcdafcdc 2023-11-27 14:56:47,755 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b756279af8c43f2b618ff51dcdafcdc 2023-11-27 14:56:47,755 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/7b756279af8c43f2b618ff51dcdafcdc, entries=1, sequenceid=23, filesize=5.0 K 2023-11-27 14:56:47,756 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d0e7e27ef1ca02c65babbede1720ba6c in 133ms, sequenceid=23, compaction requested=true 2023-11-27 14:56:47,756 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:47,756 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:47,756 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-27 14:56:48,078 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/0b25340360204254b8ea2dbb81aafdf0 2023-11-27 14:56:48,110 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b25340360204254b8ea2dbb81aafdf0 2023-11-27 14:56:48,134 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/3fabc876947e44abb9408514cb5e68da 2023-11-27 14:56:48,138 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3fabc876947e44abb9408514cb5e68da 2023-11-27 14:56:48,139 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/0b25340360204254b8ea2dbb81aafdf0 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/0b25340360204254b8ea2dbb81aafdf0 2023-11-27 14:56:48,175 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b25340360204254b8ea2dbb81aafdf0 2023-11-27 14:56:48,175 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/0b25340360204254b8ea2dbb81aafdf0, entries=3, sequenceid=23, filesize=5.2 K 2023-11-27 14:56:48,176 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/3fabc876947e44abb9408514cb5e68da as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3fabc876947e44abb9408514cb5e68da 2023-11-27 14:56:48,182 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3fabc876947e44abb9408514cb5e68da 2023-11-27 14:56:48,182 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3fabc876947e44abb9408514cb5e68da, entries=1, sequenceid=23, filesize=5.0 K 2023-11-27 14:56:48,183 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for cbfe88cf0c0ecdf504904655dc8d73d8 in 561ms, sequenceid=23, compaction requested=true 2023-11-27 14:56:48,183 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:48,183 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:48,183 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-27 14:56:48,470 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/99332f78d5de4d98a2688810ff214fbf 2023-11-27 14:56:48,493 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99332f78d5de4d98a2688810ff214fbf 2023-11-27 14:56:49,309 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/c222cf2d8db84319929840bc2f773238 2023-11-27 14:56:49,312 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c222cf2d8db84319929840bc2f773238 2023-11-27 14:56:49,313 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/99332f78d5de4d98a2688810ff214fbf as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/99332f78d5de4d98a2688810ff214fbf 2023-11-27 14:56:49,336 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99332f78d5de4d98a2688810ff214fbf 2023-11-27 14:56:49,336 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/99332f78d5de4d98a2688810ff214fbf, entries=3, sequenceid=23, filesize=5.2 K 2023-11-27 14:56:49,337 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/c222cf2d8db84319929840bc2f773238 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/c222cf2d8db84319929840bc2f773238 2023-11-27 14:56:49,340 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c222cf2d8db84319929840bc2f773238 2023-11-27 14:56:49,340 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/c222cf2d8db84319929840bc2f773238, entries=1, sequenceid=23, filesize=5.0 K 2023-11-27 14:56:49,340 INFO [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f50f095dfe928b667178fa9ffe5f7373 in 1717ms, sequenceid=23, compaction requested=true 2023-11-27 14:56:49,340 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:49,340 DEBUG [rs(jenkins-hbase4.apache.org,42669,1701096850360)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:49,341 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-27 14:56:49,341 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-27 14:56:49,341 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,341 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-27 14:56:49,341 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-27 14:56:49,342 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,342 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-27 14:56:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:49,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42669,1701096850360' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-27 14:56:49,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e018dc4[Count = 0] remaining members to acquire global barrier 2023-11-27 14:56:49,344 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-27 14:56:49,344 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:49,345 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:49,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:49,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:49,345 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-27 14:56:49,345 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-27 14:56:49,345 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42669,1701096850360' in zk 2023-11-27 14:56:49,345 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,345 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-27 14:56:49,346 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-27 14:56:49,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:49,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:49,347 DEBUG [member: 'jenkins-hbase4.apache.org,42669,1701096850360' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-27 14:56:49,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:49,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:49,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:49,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:49,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:49,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42669,1701096850360': 2023-11-27 14:56:49,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42669,1701096850360' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-27 14:56:49,348 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-27 14:56:49,348 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-27 14:56:49,348 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-27 14:56:49,349 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-27 14:56:49,349 INFO [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-27 14:56:49,351 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:49,351 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:49,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:49,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-27 14:56:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-27 14:56:49,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:49,351 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-27 14:56:49,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:49,352 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:49,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-27 14:56:49,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:49,353 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] zookeeper.ZKUtil(163): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-27 14:56:49,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-27 14:56:49,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,355 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-27 14:56:49,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-27 14:56:49,356 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-27 14:56:49,356 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-27 14:56:49,356 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:49,356 DEBUG [(jenkins-hbase4.apache.org,39835,1701096849177)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:49,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-27 14:56:49,356 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-27 14:56:49,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-27 14:56:49,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-27 14:56:49,456 DEBUG [Listener at localhost/37439] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-27 14:56:49,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39835] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-27 14:56:49,458 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:49,458 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): cbfe88cf0c0ecdf504904655dc8d73d8/family1 is initiating major compaction (all files) 2023-11-27 14:56:49,458 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:49,458 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:49,458 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of cbfe88cf0c0ecdf504904655dc8d73d8/family1 in testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:49,458 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/a3f37411a819439ea7803f577e986f98, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/9199780bf15341fcb4bc620d08a30c71, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/62cddfe482514e56a8dd75b85b7d17be, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/b2617319713f4706be4015d1fe264512, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/0b25340360204254b8ea2dbb81aafdf0] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp, totalSize=27.1 K 2023-11-27 14:56:49,459 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting a3f37411a819439ea7803f577e986f98, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701097001586 2023-11-27 14:56:49,459 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 9199780bf15341fcb4bc620d08a30c71, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701097003477 2023-11-27 14:56:49,460 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 62cddfe482514e56a8dd75b85b7d17be, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701097004528 2023-11-27 14:56:49,460 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting b2617319713f4706be4015d1fe264512, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701097006025 2023-11-27 14:56:49,460 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 0b25340360204254b8ea2dbb81aafdf0, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:49,469 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): cbfe88cf0c0ecdf504904655dc8d73d8#family1#compaction#196 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:49,925 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231127ec3cf3137a2a4cfc880c55e20a1bfc58_del 2023-11-27 14:56:49,926 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127ec3cf3137a2a4cfc880c55e20a1bfc58_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127ec3cf3137a2a4cfc880c55e20a1bfc58_del 2023-11-27 14:56:50,403 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e221e00a24e0494e81c233d76a5f3de6 2023-11-27 14:56:50,403 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family1/e221e00a24e0494e81c233d76a5f3de6 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/e221e00a24e0494e81c233d76a5f3de6 2023-11-27 14:56:50,428 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e221e00a24e0494e81c233d76a5f3de6 2023-11-27 14:56:50,429 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cbfe88cf0c0ecdf504904655dc8d73d8/family1 of cbfe88cf0c0ecdf504904655dc8d73d8 into e221e00a24e0494e81c233d76a5f3de6(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:50,429 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:50,429 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:50,430 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): cbfe88cf0c0ecdf504904655dc8d73d8/family2 is initiating major compaction (all files) 2023-11-27 14:56:50,430 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:50,430 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:50,430 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of cbfe88cf0c0ecdf504904655dc8d73d8/family2 in testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:50,430 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3e2cca46713743718603924a0b905184, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/297fc5fcdd0e4efbb8478f47f6615984, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/1e9e46c4f4534f598dbd49f23b054f4e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/924a9cf301a04f2dae845bedb50681cf, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3fabc876947e44abb9408514cb5e68da] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp, totalSize=25.4 K 2023-11-27 14:56:50,431 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 3e2cca46713743718603924a0b905184, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701097001586 2023-11-27 14:56:50,431 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 297fc5fcdd0e4efbb8478f47f6615984, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701097003477 2023-11-27 14:56:50,431 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 1e9e46c4f4534f598dbd49f23b054f4e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701097004528 2023-11-27 14:56:50,432 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 924a9cf301a04f2dae845bedb50681cf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701097006025 2023-11-27 14:56:50,432 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 3fabc876947e44abb9408514cb5e68da, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:50,449 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): cbfe88cf0c0ecdf504904655dc8d73d8#family2#compaction#197 average throughput is 0.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:50,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/a3f37411a819439ea7803f577e986f98, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/9199780bf15341fcb4bc620d08a30c71, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/62cddfe482514e56a8dd75b85b7d17be, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/b2617319713f4706be4015d1fe264512, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/0b25340360204254b8ea2dbb81aafdf0] to archive 2023-11-27 14:56:50,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:50,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/a3f37411a819439ea7803f577e986f98 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/a3f37411a819439ea7803f577e986f98 2023-11-27 14:56:50,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/9199780bf15341fcb4bc620d08a30c71 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/9199780bf15341fcb4bc620d08a30c71 2023-11-27 14:56:50,461 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/62cddfe482514e56a8dd75b85b7d17be to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/62cddfe482514e56a8dd75b85b7d17be 2023-11-27 14:56:50,465 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/b2617319713f4706be4015d1fe264512 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/b2617319713f4706be4015d1fe264512 2023-11-27 14:56:50,465 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231127d9c71565b5d94d6fa1814eca8a2c7b6c_del 2023-11-27 14:56:50,465 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127d9c71565b5d94d6fa1814eca8a2c7b6c_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231127d9c71565b5d94d6fa1814eca8a2c7b6c_del 2023-11-27 14:56:50,466 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/0b25340360204254b8ea2dbb81aafdf0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/0b25340360204254b8ea2dbb81aafdf0 2023-11-27 14:56:50,486 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b3ae73b8fe44143ace17168e1fe640f 2023-11-27 14:56:50,486 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/.tmp/family2/4b3ae73b8fe44143ace17168e1fe640f as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/4b3ae73b8fe44143ace17168e1fe640f 2023-11-27 14:56:50,490 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b3ae73b8fe44143ace17168e1fe640f 2023-11-27 14:56:50,491 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cbfe88cf0c0ecdf504904655dc8d73d8/family2 of cbfe88cf0c0ecdf504904655dc8d73d8 into 4b3ae73b8fe44143ace17168e1fe640f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:50,491 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:50,491 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:50,491 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): d0e7e27ef1ca02c65babbede1720ba6c/family1 is initiating major compaction (all files) 2023-11-27 14:56:50,492 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:50,492 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:50,495 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of d0e7e27ef1ca02c65babbede1720ba6c/family1 in testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:50,496 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/0dcd684ad4b442aa8a4151412f73daa2, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/2caab41acb6041f3a2c195075a7d93a3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/d044412080064134b550732722d6524f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/a0ea89951e6345f8af2625bfaae5f0be, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/792d6bdcabf848d1afbaecba533182c1] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp, totalSize=27.1 K 2023-11-27 14:56:50,499 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 0dcd684ad4b442aa8a4151412f73daa2, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701097001587 2023-11-27 14:56:50,499 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 2caab41acb6041f3a2c195075a7d93a3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701097003477 2023-11-27 14:56:50,500 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting d044412080064134b550732722d6524f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701097004528 2023-11-27 14:56:50,500 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting a0ea89951e6345f8af2625bfaae5f0be, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701097006025 2023-11-27 14:56:50,501 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 792d6bdcabf848d1afbaecba533182c1, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:50,521 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): d0e7e27ef1ca02c65babbede1720ba6c#family1#compaction#198 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:50,554 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3e2cca46713743718603924a0b905184, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/297fc5fcdd0e4efbb8478f47f6615984, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/1e9e46c4f4534f598dbd49f23b054f4e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/924a9cf301a04f2dae845bedb50681cf, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3fabc876947e44abb9408514cb5e68da] to archive 2023-11-27 14:56:50,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:50,559 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3e2cca46713743718603924a0b905184 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3e2cca46713743718603924a0b905184 2023-11-27 14:56:50,560 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/297fc5fcdd0e4efbb8478f47f6615984 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/297fc5fcdd0e4efbb8478f47f6615984 2023-11-27 14:56:50,562 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/1e9e46c4f4534f598dbd49f23b054f4e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/1e9e46c4f4534f598dbd49f23b054f4e 2023-11-27 14:56:50,564 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/924a9cf301a04f2dae845bedb50681cf to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/924a9cf301a04f2dae845bedb50681cf 2023-11-27 14:56:50,565 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3fabc876947e44abb9408514cb5e68da to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family2/3fabc876947e44abb9408514cb5e68da 2023-11-27 14:56:50,593 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311272a23ee5d2856455aa39cacdfbd6646db_del 2023-11-27 14:56:50,593 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311272a23ee5d2856455aa39cacdfbd6646db_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311272a23ee5d2856455aa39cacdfbd6646db_del 2023-11-27 14:56:50,681 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2d9d89b895d14a67b027687ce6bba4cf 2023-11-27 14:56:50,682 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family1/2d9d89b895d14a67b027687ce6bba4cf as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/2d9d89b895d14a67b027687ce6bba4cf 2023-11-27 14:56:50,717 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2d9d89b895d14a67b027687ce6bba4cf 2023-11-27 14:56:50,718 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d0e7e27ef1ca02c65babbede1720ba6c/family1 of d0e7e27ef1ca02c65babbede1720ba6c into 2d9d89b895d14a67b027687ce6bba4cf(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:50,718 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:50,718 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:50,718 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): d0e7e27ef1ca02c65babbede1720ba6c/family2 is initiating major compaction (all files) 2023-11-27 14:56:50,718 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:50,719 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:50,719 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of d0e7e27ef1ca02c65babbede1720ba6c/family2 in testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:50,719 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/0f8bce2c6e624154abdedddb63d96ead, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/1f89e202540a44a691c8ddcb9b88b836, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/303a8053de724ae4a6243745bc2a18ca, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/de8d5800d9c947b0b4205175ed7e8db6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/7b756279af8c43f2b618ff51dcdafcdc] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp, totalSize=25.4 K 2023-11-27 14:56:50,719 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 0f8bce2c6e624154abdedddb63d96ead, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701097001587 2023-11-27 14:56:50,720 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 1f89e202540a44a691c8ddcb9b88b836, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701097003477 2023-11-27 14:56:50,720 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 303a8053de724ae4a6243745bc2a18ca, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701097004528 2023-11-27 14:56:50,721 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting de8d5800d9c947b0b4205175ed7e8db6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701097006025 2023-11-27 14:56:50,721 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 7b756279af8c43f2b618ff51dcdafcdc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:50,737 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): d0e7e27ef1ca02c65babbede1720ba6c#family2#compaction#199 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:50,751 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/0dcd684ad4b442aa8a4151412f73daa2, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/2caab41acb6041f3a2c195075a7d93a3, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/d044412080064134b550732722d6524f, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/a0ea89951e6345f8af2625bfaae5f0be, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/792d6bdcabf848d1afbaecba533182c1] to archive 2023-11-27 14:56:50,752 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:50,754 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/0dcd684ad4b442aa8a4151412f73daa2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/0dcd684ad4b442aa8a4151412f73daa2 2023-11-27 14:56:50,755 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/2caab41acb6041f3a2c195075a7d93a3 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/2caab41acb6041f3a2c195075a7d93a3 2023-11-27 14:56:50,757 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/d044412080064134b550732722d6524f to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/d044412080064134b550732722d6524f 2023-11-27 14:56:50,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/a0ea89951e6345f8af2625bfaae5f0be to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/a0ea89951e6345f8af2625bfaae5f0be 2023-11-27 14:56:50,760 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/792d6bdcabf848d1afbaecba533182c1 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/792d6bdcabf848d1afbaecba533182c1 2023-11-27 14:56:50,763 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231127aff1f781dded4b4e8462d3bf3f74adae_del 2023-11-27 14:56:50,763 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127aff1f781dded4b4e8462d3bf3f74adae_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231127aff1f781dded4b4e8462d3bf3f74adae_del 2023-11-27 14:56:51,577 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 629af9ead558492bbf3699915b285fb8 2023-11-27 14:56:51,578 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/.tmp/family2/629af9ead558492bbf3699915b285fb8 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/629af9ead558492bbf3699915b285fb8 2023-11-27 14:56:51,581 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 629af9ead558492bbf3699915b285fb8 2023-11-27 14:56:51,582 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d0e7e27ef1ca02c65babbede1720ba6c/family2 of d0e7e27ef1ca02c65babbede1720ba6c into 629af9ead558492bbf3699915b285fb8(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-27 14:56:51,582 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:51,582 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:51,582 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): f50f095dfe928b667178fa9ffe5f7373/family1 is initiating major compaction (all files) 2023-11-27 14:56:51,582 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:51,582 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:51,583 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of f50f095dfe928b667178fa9ffe5f7373/family1 in testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:51,583 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/38016a82c820476399e57e6d2a9c2a08, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/28b67c6724324952b51fb3cb80202eb2, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/cc6d594b87044fec864bb91908867cac, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/2be0e6f3e12f4021954b97b4739ec2cf, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/99332f78d5de4d98a2688810ff214fbf] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp, totalSize=27.1 K 2023-11-27 14:56:51,583 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 38016a82c820476399e57e6d2a9c2a08, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701097001587 2023-11-27 14:56:51,583 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 28b67c6724324952b51fb3cb80202eb2, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701097003477 2023-11-27 14:56:51,584 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting cc6d594b87044fec864bb91908867cac, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701097004528 2023-11-27 14:56:51,584 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 2be0e6f3e12f4021954b97b4739ec2cf, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701097006025 2023-11-27 14:56:51,584 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 99332f78d5de4d98a2688810ff214fbf, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:51,591 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): f50f095dfe928b667178fa9ffe5f7373#family1#compaction#200 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:51,661 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/0f8bce2c6e624154abdedddb63d96ead, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/1f89e202540a44a691c8ddcb9b88b836, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/303a8053de724ae4a6243745bc2a18ca, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/de8d5800d9c947b0b4205175ed7e8db6, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/7b756279af8c43f2b618ff51dcdafcdc] to archive 2023-11-27 14:56:51,662 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:51,663 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/0f8bce2c6e624154abdedddb63d96ead to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/0f8bce2c6e624154abdedddb63d96ead 2023-11-27 14:56:51,664 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/1f89e202540a44a691c8ddcb9b88b836 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/1f89e202540a44a691c8ddcb9b88b836 2023-11-27 14:56:51,665 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/303a8053de724ae4a6243745bc2a18ca to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/303a8053de724ae4a6243745bc2a18ca 2023-11-27 14:56:51,666 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/de8d5800d9c947b0b4205175ed7e8db6 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/de8d5800d9c947b0b4205175ed7e8db6 2023-11-27 14:56:51,667 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/7b756279af8c43f2b618ff51dcdafcdc to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family2/7b756279af8c43f2b618ff51dcdafcdc 2023-11-27 14:56:52,030 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231127f1ec9031cce0497a88e8aa3ff5d116d8_del 2023-11-27 14:56:52,031 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127f1ec9031cce0497a88e8aa3ff5d116d8_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127f1ec9031cce0497a88e8aa3ff5d116d8_del 2023-11-27 14:56:52,885 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 268bd6b5596448039ea6f09504e88cf1 2023-11-27 14:56:52,886 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family1/268bd6b5596448039ea6f09504e88cf1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/268bd6b5596448039ea6f09504e88cf1 2023-11-27 14:56:52,908 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 268bd6b5596448039ea6f09504e88cf1 2023-11-27 14:56:52,909 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f50f095dfe928b667178fa9ffe5f7373/family1 of f50f095dfe928b667178fa9ffe5f7373 into 268bd6b5596448039ea6f09504e88cf1(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:52,909 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:52,909 DEBUG [Listener at localhost/37439] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-27 14:56:52,909 DEBUG [Listener at localhost/37439] regionserver.HStore(1912): f50f095dfe928b667178fa9ffe5f7373/family2 is initiating major compaction (all files) 2023-11-27 14:56:52,909 INFO [Listener at localhost/37439] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-27 14:56:52,909 INFO [Listener at localhost/37439] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-27 14:56:52,909 INFO [Listener at localhost/37439] regionserver.HRegion(2259): Starting compaction of f50f095dfe928b667178fa9ffe5f7373/family2 in testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:52,909 INFO [Listener at localhost/37439] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/80bb038a61654a3a9b25be75bb232d9e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/9e7a5e86abde458d9e3aa8f0c17b4311, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/f2e7c42c3925472999717f975a339156, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/a7cea3e5333246ce93fa02a35951eabf, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/c222cf2d8db84319929840bc2f773238] into tmpdir=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp, totalSize=25.4 K 2023-11-27 14:56:52,910 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 80bb038a61654a3a9b25be75bb232d9e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1701097001587 2023-11-27 14:56:52,910 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting 9e7a5e86abde458d9e3aa8f0c17b4311, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1701097003477 2023-11-27 14:56:52,910 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting f2e7c42c3925472999717f975a339156, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1701097004528 2023-11-27 14:56:52,911 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting a7cea3e5333246ce93fa02a35951eabf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1701097006025 2023-11-27 14:56:52,911 DEBUG [Listener at localhost/37439] compactions.Compactor(207): Compacting c222cf2d8db84319929840bc2f773238, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-27 14:56:52,918 INFO [Listener at localhost/37439] throttle.PressureAwareThroughputController(145): f50f095dfe928b667178fa9ffe5f7373#family2#compaction#201 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-27 14:56:52,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/38016a82c820476399e57e6d2a9c2a08, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/28b67c6724324952b51fb3cb80202eb2, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/cc6d594b87044fec864bb91908867cac, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/2be0e6f3e12f4021954b97b4739ec2cf, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/99332f78d5de4d98a2688810ff214fbf] to archive 2023-11-27 14:56:52,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:52,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/38016a82c820476399e57e6d2a9c2a08 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/38016a82c820476399e57e6d2a9c2a08 2023-11-27 14:56:52,954 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/28b67c6724324952b51fb3cb80202eb2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/28b67c6724324952b51fb3cb80202eb2 2023-11-27 14:56:52,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/cc6d594b87044fec864bb91908867cac to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/cc6d594b87044fec864bb91908867cac 2023-11-27 14:56:52,956 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/2be0e6f3e12f4021954b97b4739ec2cf to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/2be0e6f3e12f4021954b97b4739ec2cf 2023-11-27 14:56:52,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/99332f78d5de4d98a2688810ff214fbf to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/99332f78d5de4d98a2688810ff214fbf 2023-11-27 14:56:53,325 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311273694b593dcea46b88f12fc78c1adfbf8_del 2023-11-27 14:56:53,326 INFO [Listener at localhost/37439] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311273694b593dcea46b88f12fc78c1adfbf8_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202311273694b593dcea46b88f12fc78c1adfbf8_del 2023-11-27 14:56:54,137 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 884e572dfb4849eeb191d05589ad6adb 2023-11-27 14:56:54,138 DEBUG [Listener at localhost/37439] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/.tmp/family2/884e572dfb4849eeb191d05589ad6adb as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/884e572dfb4849eeb191d05589ad6adb 2023-11-27 14:56:54,142 INFO [Listener at localhost/37439] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 884e572dfb4849eeb191d05589ad6adb 2023-11-27 14:56:54,143 INFO [Listener at localhost/37439] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f50f095dfe928b667178fa9ffe5f7373/family2 of f50f095dfe928b667178fa9ffe5f7373 into 884e572dfb4849eeb191d05589ad6adb(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-27 14:56:54,143 DEBUG [Listener at localhost/37439] regionserver.HRegion(2289): Compaction status journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:54,146 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:54,148 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:54,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/80bb038a61654a3a9b25be75bb232d9e, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/9e7a5e86abde458d9e3aa8f0c17b4311, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/f2e7c42c3925472999717f975a339156, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/a7cea3e5333246ce93fa02a35951eabf, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/c222cf2d8db84319929840bc2f773238] to archive 2023-11-27 14:56:54,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:54,152 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/80bb038a61654a3a9b25be75bb232d9e to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/80bb038a61654a3a9b25be75bb232d9e 2023-11-27 14:56:54,152 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:54,153 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:54,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/9e7a5e86abde458d9e3aa8f0c17b4311 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/9e7a5e86abde458d9e3aa8f0c17b4311 2023-11-27 14:56:54,154 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/f2e7c42c3925472999717f975a339156 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/f2e7c42c3925472999717f975a339156 2023-11-27 14:56:54,155 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/a7cea3e5333246ce93fa02a35951eabf to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/a7cea3e5333246ce93fa02a35951eabf 2023-11-27 14:56:54,157 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/c222cf2d8db84319929840bc2f773238 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family2/c222cf2d8db84319929840bc2f773238 2023-11-27 14:56:54,158 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39835] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10={\xB2\x8A\x8C\xB2\x01(\x1D\xDB\xAFa\x00\x95W\xD9"\x10\x9E\xC9+.]t\xF6\xFC;K\xEE\xB9\x0D\x15u\x94*\x10B\x09\x0B\xEA\xDB\x1C\xEC%[4Te\x1F\xD7\x0D\x822\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-11-27 14:56:54,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] procedure2.ProcedureExecutor(1028): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-27 14:56:54,181 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-27 14:56:54,182 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:56:54,182 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1701097614182 2023-11-27 14:56:54,182 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-27 14:56:54,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-11-27 14:56:54,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231127ec3cf3137a2a4cfc880c55e20a1bfc58_del 2023-11-27 14:56:54,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311272a23ee5d2856455aa39cacdfbd6646db_del 2023-11-27 14:56:54,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231127f1ec9031cce0497a88e8aa3ff5d116d8_del 2023-11-27 14:56:54,558 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-11-27 14:56:54,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] zookeeper.ReadOnlyZKClient(139): Connect 0x59601b6f to 127.0.0.1:61098 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-27 14:56:54,564 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@67be7296, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-27 14:56:54,564 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231127 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-27 14:56:55,120 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231127c081305bfcaf4c30a92e7dfdc08ccf29 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127c081305bfcaf4c30a92e7dfdc08ccf29 2023-11-27 14:56:55,121 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='testMajorCompactionFromAdmin' column='family1' 2023-11-27 14:56:55,136 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-27 14:56:55,138 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59430, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-27 14:56:55,150 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-27 14:56:55,151 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42130, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-27 14:56:55,191 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231127/family1/be9c3b588cdf4c918f08b69f4c1d07d5 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-11-27 14:56:55,192 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231127/family1/be9c3b588cdf4c918f08b69f4c1d07d5}] 2023-11-27 14:56:55,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231127/family1/be9c3b588cdf4c918f08b69f4c1d07d5 for inclusion in cbfe88cf0c0ecdf504904655dc8d73d8/family1 2023-11-27 14:56:55,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-11-27 14:56:55,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first= last=1 2023-11-27 14:56:55,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:55,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231127/family1/be9c3b588cdf4c918f08b69f4c1d07d5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__ufcgco88ib0b5disgi6a8rqtictsetu0bul9ocjru2glhpgpic4ufumn93vd9ss9/family1/be9c3b588cdf4c918f08b69f4c1d07d5 2023-11-27 14:56:55,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__ufcgco88ib0b5disgi6a8rqtictsetu0bul9ocjru2glhpgpic4ufumn93vd9ss9/family1/be9c3b588cdf4c918f08b69f4c1d07d5 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/f0e04caf59b841f98155113f73c216a2_SeqId_28_ 2023-11-27 14:56:55,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__ufcgco88ib0b5disgi6a8rqtictsetu0bul9ocjru2glhpgpic4ufumn93vd9ss9/family1/be9c3b588cdf4c918f08b69f4c1d07d5 into cbfe88cf0c0ecdf504904655dc8d73d8/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/f0e04caf59b841f98155113f73c216a2_SeqId_28_ - updating store file list. 2023-11-27 14:56:55,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/f0e04caf59b841f98155113f73c216a2_SeqId_28_ into cbfe88cf0c0ecdf504904655dc8d73d8/family1 2023-11-27 14:56:55,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__ufcgco88ib0b5disgi6a8rqtictsetu0bul9ocjru2glhpgpic4ufumn93vd9ss9/family1/be9c3b588cdf4c918f08b69f4c1d07d5 into cbfe88cf0c0ecdf504904655dc8d73d8/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/family1/f0e04caf59b841f98155113f73c216a2_SeqId_28_) 2023-11-27 14:56:55,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__ufcgco88ib0b5disgi6a8rqtictsetu0bul9ocjru2glhpgpic4ufumn93vd9ss9/family1/be9c3b588cdf4c918f08b69f4c1d07d5 2023-11-27 14:56:55,256 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231127 table='testMajorCompactionFromAdmin' column='family1' 2023-11-27 14:56:55,257 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:55,258 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023112703e33662836146478d1f804b6eea6c93 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023112703e33662836146478d1f804b6eea6c93 2023-11-27 14:56:55,259 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202311275c5d201cff814f068d1cf190fc6d0a27 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202311275c5d201cff814f068d1cf190fc6d0a27 2023-11-27 14:56:55,261 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127643be05eb02748b6ad284bdb40fb6dd2 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127643be05eb02748b6ad284bdb40fb6dd2 2023-11-27 14:56:55,262 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127daaf643b2e1d4646a0dcbbae6d8704c8 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127daaf643b2e1d4646a0dcbbae6d8704c8 2023-11-27 14:56:55,262 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-27 14:56:55,262 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231127' 2023-11-27 14:56:55,263 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231127 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-27 14:56:55,263 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231127 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-27 14:56:55,876 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231127b3124535ecf641a89087410d76b6520a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231127b3124535ecf641a89087410d76b6520a 2023-11-27 14:56:55,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231127 table='testMajorCompactionFromAdmin' column='family1' 2023-11-27 14:56:55,922 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231127/family1/402bf99ff99c414788180122b02b9d21 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-11-27 14:56:55,922 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231127/family1/402bf99ff99c414788180122b02b9d21}] 2023-11-27 14:56:55,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231127/family1/402bf99ff99c414788180122b02b9d21 for inclusion in d0e7e27ef1ca02c65babbede1720ba6c/family1 2023-11-27 14:56:55,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-11-27 14:56:55,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-27 14:56:55,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:55,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231127/family1/402bf99ff99c414788180122b02b9d21 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__8ru8qmbc97d95u2neggci5qsb3fp3fl4b5sgsd6n8hke89ie9tjqarmudla0k0pj/family1/402bf99ff99c414788180122b02b9d21 2023-11-27 14:56:55,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__8ru8qmbc97d95u2neggci5qsb3fp3fl4b5sgsd6n8hke89ie9tjqarmudla0k0pj/family1/402bf99ff99c414788180122b02b9d21 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/5082585bae3c4ae5943624cc9550058a_SeqId_28_ 2023-11-27 14:56:55,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__8ru8qmbc97d95u2neggci5qsb3fp3fl4b5sgsd6n8hke89ie9tjqarmudla0k0pj/family1/402bf99ff99c414788180122b02b9d21 into d0e7e27ef1ca02c65babbede1720ba6c/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/5082585bae3c4ae5943624cc9550058a_SeqId_28_ - updating store file list. 2023-11-27 14:56:55,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/5082585bae3c4ae5943624cc9550058a_SeqId_28_ into d0e7e27ef1ca02c65babbede1720ba6c/family1 2023-11-27 14:56:55,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__8ru8qmbc97d95u2neggci5qsb3fp3fl4b5sgsd6n8hke89ie9tjqarmudla0k0pj/family1/402bf99ff99c414788180122b02b9d21 into d0e7e27ef1ca02c65babbede1720ba6c/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/family1/5082585bae3c4ae5943624cc9550058a_SeqId_28_) 2023-11-27 14:56:55,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__8ru8qmbc97d95u2neggci5qsb3fp3fl4b5sgsd6n8hke89ie9tjqarmudla0k0pj/family1/402bf99ff99c414788180122b02b9d21 2023-11-27 14:56:55,968 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231127 table='testMajorCompactionFromAdmin' column='family1' 2023-11-27 14:56:55,968 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:55,970 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311270f99d1b9081749a88baad2be396a308a to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311270f99d1b9081749a88baad2be396a308a 2023-11-27 14:56:55,970 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231127360161170fb941f99c4e80f31cb0ff14 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231127360161170fb941f99c4e80f31cb0ff14 2023-11-27 14:56:55,971 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311277fae167b78c74157b7f442397bcd5bea to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311277fae167b78c74157b7f442397bcd5bea 2023-11-27 14:56:55,972 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231127b987fba0c754400ea5e7f7262f6be892 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231127b987fba0c754400ea5e7f7262f6be892 2023-11-27 14:56:55,972 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-27 14:56:55,972 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231127' 2023-11-27 14:56:55,972 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231127 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-27 14:56:55,972 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231127 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-27 14:56:56,555 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231127e8a95aa065e045ef841de044845d8b32 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127e8a95aa065e045ef841de044845d8b32 2023-11-27 14:56:56,556 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='testMajorCompactionFromAdmin' column='family1' 2023-11-27 14:56:56,594 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231127/family1/7a80b8287bab477d85c3b9d5e87e21a5 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-11-27 14:56:56,595 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373., hostname=jenkins-hbase4.apache.org,42669,1701096850360, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231127/family1/7a80b8287bab477d85c3b9d5e87e21a5}] 2023-11-27 14:56:56,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(763): Validating hfile at hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231127/family1/7a80b8287bab477d85c3b9d5e87e21a5 for inclusion in f50f095dfe928b667178fa9ffe5f7373/family1 2023-11-27 14:56:56,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-11-27 14:56:56,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-27 14:56:56,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegion(2446): Flush status journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:56,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231127/family1/7a80b8287bab477d85c3b9d5e87e21a5 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__i6adoccu85k102b6g5umsal3pmbq5c217el450jn5ljh3u27knts8gj5p8dhnb64/family1/7a80b8287bab477d85c3b9d5e87e21a5 2023-11-27 14:56:56,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__i6adoccu85k102b6g5umsal3pmbq5c217el450jn5ljh3u27knts8gj5p8dhnb64/family1/7a80b8287bab477d85c3b9d5e87e21a5 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/9d3e6aee7d6048d5a8962fc8ed95f504_SeqId_28_ 2023-11-27 14:56:56,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(849): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__i6adoccu85k102b6g5umsal3pmbq5c217el450jn5ljh3u27knts8gj5p8dhnb64/family1/7a80b8287bab477d85c3b9d5e87e21a5 into f50f095dfe928b667178fa9ffe5f7373/family1 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/9d3e6aee7d6048d5a8962fc8ed95f504_SeqId_28_ - updating store file list. 2023-11-27 14:56:56,640 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(882): Loaded HFile hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/9d3e6aee7d6048d5a8962fc8ed95f504_SeqId_28_ into f50f095dfe928b667178fa9ffe5f7373/family1 2023-11-27 14:56:56,640 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.HStore(855): Successfully loaded hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__i6adoccu85k102b6g5umsal3pmbq5c217el450jn5ljh3u27knts8gj5p8dhnb64/family1/7a80b8287bab477d85c3b9d5e87e21a5 into f50f095dfe928b667178fa9ffe5f7373/family1 (new location: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/family1/9d3e6aee7d6048d5a8962fc8ed95f504_SeqId_28_) 2023-11-27 14:56:56,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42669] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/staging/jenkins__testMajorCompactionFromAdmin__i6adoccu85k102b6g5umsal3pmbq5c217el450jn5ljh3u27knts8gj5p8dhnb64/family1/7a80b8287bab477d85c3b9d5e87e21a5 2023-11-27 14:56:56,642 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231127 table='testMajorCompactionFromAdmin' column='family1' 2023-11-27 14:56:56,643 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:56,644 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127035ecc4513c74ab3b0a0414e80cf1fd8 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127035ecc4513c74ab3b0a0414e80cf1fd8 2023-11-27 14:56:56,645 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311272f08d1fada274a58af398be2e3c8809b to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311272f08d1fada274a58af398be2e3c8809b 2023-11-27 14:56:56,645 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127a8a3893c33044c44b17cbb70bfd1c2e0 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127a8a3893c33044c44b17cbb70bfd1c2e0 2023-11-27 14:56:56,646 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127f7bc58594b944fe787577aca64ad6e61 to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127f7bc58594b944fe787577aca64ad6e61 2023-11-27 14:56:56,646 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-27 14:56:56,646 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231127' 2023-11-27 14:56:56,646 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231127 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-27 14:56:56,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-11-27 14:56:56,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-27 14:56:56,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(367): [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127ec3cf3137a2a4cfc880c55e20a1bfc58_del] 2023-11-27 14:56:56,647 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:56,648 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127ec3cf3137a2a4cfc880c55e20a1bfc58_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231127ec3cf3137a2a4cfc880c55e20a1bfc58_del 2023-11-27 14:56:56,648 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-27 14:56:56,648 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(367): [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311272a23ee5d2856455aa39cacdfbd6646db_del] 2023-11-27 14:56:56,648 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:56,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311272a23ee5d2856455aa39cacdfbd6646db_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311272a23ee5d2856455aa39cacdfbd6646db_del 2023-11-27 14:56:56,649 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-27 14:56:56,649 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] compactions.PartitionedMobCompactor(367): [hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127f1ec9031cce0497a88e8aa3ff5d116d8_del] 2023-11-27 14:56:56,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] backup.HFileArchiver(360): Archiving compacted files. 2023-11-27 14:56:56,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127f1ec9031cce0497a88e8aa3ff5d116d8_del to hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231127f1ec9031cce0497a88e8aa3ff5d116d8_del 2023-11-27 14:56:56,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-27 14:56:56,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1701096853170] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-27 14:56:56,652 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-27 14:56:56,653 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.4720 sec 2023-11-27 14:56:56,673 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:56,675 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:56,677 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-27 14:56:56,677 DEBUG [Listener at localhost/37439] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-27 14:56:56,680 INFO [Listener at localhost/37439] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-27 14:56:56,714 INFO [Listener at localhost/37439] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=252 (was 266), OpenFileDescriptor=677 (was 689), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=167 (was 139) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=7382 (was 7990) 2023-11-27 14:56:56,715 DEBUG [Listener at localhost/37439] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x65c25ebc to 127.0.0.1:61098 2023-11-27 14:56:56,715 DEBUG [Listener at localhost/37439] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:56,715 INFO [Listener at localhost/37439] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-11-27 14:56:56,715 INFO [Listener at localhost/37439] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-11-27 14:56:56,715 DEBUG [Listener at localhost/37439] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x36d9c5a6 to 127.0.0.1:61098 2023-11-27 14:56:56,715 DEBUG [Listener at localhost/37439] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:56,716 DEBUG [Listener at localhost/37439] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-11-27 14:56:56,716 DEBUG [Listener at localhost/37439] util.JVMClusterUtil(257): Found active master hash=1815401014, stopped=false 2023-11-27 14:56:56,716 INFO [Listener at localhost/37439] master.ServerManager(888): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:56:56,717 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-27 14:56:56,717 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-27 14:56:56,717 INFO [Listener at localhost/37439] procedure2.ProcedureExecutor(628): Stopping 2023-11-27 14:56:56,717 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:56:56,718 DEBUG [Listener at localhost/37439] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x55df12ba to 127.0.0.1:61098 2023-11-27 14:56:56,718 DEBUG [Listener at localhost/37439] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:56,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-27 14:56:56,718 INFO [Listener at localhost/37439] regionserver.HRegionServer(2300): ***** STOPPING region server 'jenkins-hbase4.apache.org,42669,1701096850360' ***** 2023-11-27 14:56:56,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-27 14:56:56,718 INFO [Listener at localhost/37439] regionserver.HRegionServer(2314): STOPPED: Shutdown requested 2023-11-27 14:56:56,718 INFO [RS:0;jenkins-hbase4:42669] regionserver.HeapMemoryManager(220): Stopping 2023-11-27 14:56:56,718 INFO [RS:0;jenkins-hbase4:42669] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-11-27 14:56:56,719 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 686970f6b8c01e84ad20c72c7cb3698f 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 355881cf77b1bf9999fb7e0bd0dc8e2a 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 39f113e3442d78447a6f24b4b73006c8 2023-11-27 14:56:56,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 686970f6b8c01e84ad20c72c7cb3698f, disabling compactions & flushes 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 3404a93afc957274c1f21854ed231972 2023-11-27 14:56:56,719 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for f53154fc29f582902661898287c317b5 2023-11-27 14:56:56,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for e13a129f6f563cfb251e64b3f5d12815 2023-11-27 14:56:56,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. after waiting 0 ms 2023-11-27 14:56:56,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 480bebe747409b1ccbbfa174b76957d7 2023-11-27 14:56:56,719 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 686970f6b8c01e84ad20c72c7cb3698f 1/1 column families, dataSize=109 B heapSize=592 B 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 56c85fb3659c83b26d25644563d1ffc6 2023-11-27 14:56:56,719 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 1cf135bbee78d983093ed681ad9bc867 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for b70da5c01d91e889a747889289d215f3 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 36cece40383490f5f6bb34f20558c5b9 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 73f02d3ec84561cf82fbd52d7e1039a0 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for cbfe88cf0c0ecdf504904655dc8d73d8 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for d2db448818547f8516ae9f9e43b817b6 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 68bb86a7752ed0247b952754e90231e7 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for d7e7b1527d4ce274f466396dcf7e3231 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for d0e7e27ef1ca02c65babbede1720ba6c 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 4e630d9c8bdef52b2f9df3adf2c55277 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 8ae027c1211c228b679773750f57d6ba 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for f50f095dfe928b667178fa9ffe5f7373 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1147): stopping server jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:56,720 DEBUG [RS:0;jenkins-hbase4:42669] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4085df3f to 127.0.0.1:61098 2023-11-27 14:56:56,720 DEBUG [RS:0;jenkins-hbase4:42669] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-11-27 14:56:56,720 INFO [RS:0;jenkins-hbase4:42669] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-11-27 14:56:56,721 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(3308): Received CLOSE for 1588230740 2023-11-27 14:56:56,721 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1477): Waiting on 21 regions to close 2023-11-27 14:56:56,721 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-27 14:56:56,721 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-27 14:56:56,721 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-27 14:56:56,721 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-27 14:56:56,721 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-27 14:56:56,721 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-11-27 14:56:56,721 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1507): Waiting on 1588230740, 1cf135bbee78d983093ed681ad9bc867, 3404a93afc957274c1f21854ed231972, 355881cf77b1bf9999fb7e0bd0dc8e2a, 36cece40383490f5f6bb34f20558c5b9, 39f113e3442d78447a6f24b4b73006c8, 480bebe747409b1ccbbfa174b76957d7, 4e630d9c8bdef52b2f9df3adf2c55277, 56c85fb3659c83b26d25644563d1ffc6, 686970f6b8c01e84ad20c72c7cb3698f, 68bb86a7752ed0247b952754e90231e7, 73f02d3ec84561cf82fbd52d7e1039a0, 8ae027c1211c228b679773750f57d6ba, b70da5c01d91e889a747889289d215f3, cbfe88cf0c0ecdf504904655dc8d73d8, d0e7e27ef1ca02c65babbede1720ba6c, d2db448818547f8516ae9f9e43b817b6, d7e7b1527d4ce274f466396dcf7e3231, e13a129f6f563cfb251e64b3f5d12815, f50f095dfe928b667178fa9ffe5f7373, f53154fc29f582902661898287c317b5 2023-11-27 14:56:56,738 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/.tmp/info/94e776aa645b4ae6aeab898461363980 2023-11-27 14:56:56,741 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 94e776aa645b4ae6aeab898461363980 2023-11-27 14:56:56,746 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/.tmp/rep_barrier/302d667375644e30b084cdc5b57c645a 2023-11-27 14:56:56,749 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 302d667375644e30b084cdc5b57c645a 2023-11-27 14:56:56,749 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-11-27 14:56:56,754 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/.tmp/table/88fb21ad13104ce3b04dbe0e8bdcfb33 2023-11-27 14:56:56,756 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88fb21ad13104ce3b04dbe0e8bdcfb33 2023-11-27 14:56:56,756 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/.tmp/info/94e776aa645b4ae6aeab898461363980 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/info/94e776aa645b4ae6aeab898461363980 2023-11-27 14:56:56,759 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 94e776aa645b4ae6aeab898461363980 2023-11-27 14:56:56,759 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/info/94e776aa645b4ae6aeab898461363980, entries=226, sequenceid=130, filesize=37.9 K 2023-11-27 14:56:56,760 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/.tmp/rep_barrier/302d667375644e30b084cdc5b57c645a as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/rep_barrier/302d667375644e30b084cdc5b57c645a 2023-11-27 14:56:56,763 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 302d667375644e30b084cdc5b57c645a 2023-11-27 14:56:56,763 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/rep_barrier/302d667375644e30b084cdc5b57c645a, entries=1, sequenceid=130, filesize=5.0 K 2023-11-27 14:56:56,763 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/.tmp/table/88fb21ad13104ce3b04dbe0e8bdcfb33 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/table/88fb21ad13104ce3b04dbe0e8bdcfb33 2023-11-27 14:56:56,766 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88fb21ad13104ce3b04dbe0e8bdcfb33 2023-11-27 14:56:56,766 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/table/88fb21ad13104ce3b04dbe0e8bdcfb33, entries=27, sequenceid=130, filesize=6.6 K 2023-11-27 14:56:56,766 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 45ms, sequenceid=130, compaction requested=false 2023-11-27 14:56:56,769 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-11-27 14:56:56,770 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-27 14:56:56,770 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-27 14:56:56,770 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-27 14:56:56,771 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-11-27 14:56:56,815 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-11-27 14:56:56,922 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1507): Waiting on 1cf135bbee78d983093ed681ad9bc867, 3404a93afc957274c1f21854ed231972, 355881cf77b1bf9999fb7e0bd0dc8e2a, 36cece40383490f5f6bb34f20558c5b9, 39f113e3442d78447a6f24b4b73006c8, 480bebe747409b1ccbbfa174b76957d7, 4e630d9c8bdef52b2f9df3adf2c55277, 56c85fb3659c83b26d25644563d1ffc6, 686970f6b8c01e84ad20c72c7cb3698f, 68bb86a7752ed0247b952754e90231e7, 73f02d3ec84561cf82fbd52d7e1039a0, 8ae027c1211c228b679773750f57d6ba, b70da5c01d91e889a747889289d215f3, cbfe88cf0c0ecdf504904655dc8d73d8, d0e7e27ef1ca02c65babbede1720ba6c, d2db448818547f8516ae9f9e43b817b6, d7e7b1527d4ce274f466396dcf7e3231, e13a129f6f563cfb251e64b3f5d12815, f50f095dfe928b667178fa9ffe5f7373, f53154fc29f582902661898287c317b5 2023-11-27 14:56:57,122 DEBUG [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1507): Waiting on 1cf135bbee78d983093ed681ad9bc867, 3404a93afc957274c1f21854ed231972, 355881cf77b1bf9999fb7e0bd0dc8e2a, 36cece40383490f5f6bb34f20558c5b9, 39f113e3442d78447a6f24b4b73006c8, 480bebe747409b1ccbbfa174b76957d7, 4e630d9c8bdef52b2f9df3adf2c55277, 56c85fb3659c83b26d25644563d1ffc6, 686970f6b8c01e84ad20c72c7cb3698f, 68bb86a7752ed0247b952754e90231e7, 73f02d3ec84561cf82fbd52d7e1039a0, 8ae027c1211c228b679773750f57d6ba, b70da5c01d91e889a747889289d215f3, cbfe88cf0c0ecdf504904655dc8d73d8, d0e7e27ef1ca02c65babbede1720ba6c, d2db448818547f8516ae9f9e43b817b6, d7e7b1527d4ce274f466396dcf7e3231, e13a129f6f563cfb251e64b3f5d12815, f50f095dfe928b667178fa9ffe5f7373, f53154fc29f582902661898287c317b5 2023-11-27 14:56:57,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f/.tmp/info/0ae69f6787f64de5b2fd120e7c43f311 2023-11-27 14:56:57,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f/.tmp/info/0ae69f6787f64de5b2fd120e7c43f311 as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f/info/0ae69f6787f64de5b2fd120e7c43f311 2023-11-27 14:56:57,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f/info/0ae69f6787f64de5b2fd120e7c43f311, entries=3, sequenceid=7, filesize=4.8 K 2023-11-27 14:56:57,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 686970f6b8c01e84ad20c72c7cb3698f in 427ms, sequenceid=7, compaction requested=false 2023-11-27 14:56:57,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/hbase/namespace/686970f6b8c01e84ad20c72c7cb3698f/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-11-27 14:56:57,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:56:57,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 686970f6b8c01e84ad20c72c7cb3698f: 2023-11-27 14:56:57,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1701096852147.686970f6b8c01e84ad20c72c7cb3698f. 2023-11-27 14:56:57,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 355881cf77b1bf9999fb7e0bd0dc8e2a, disabling compactions & flushes 2023-11-27 14:56:57,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:56:57,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:56:57,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. after waiting 0 ms 2023-11-27 14:56:57,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:56:57,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithMonthlyPolicy/355881cf77b1bf9999fb7e0bd0dc8e2a/recovered.edits/51.seqid, newMaxSeqId=51, maxSeqId=1 2023-11-27 14:56:57,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:56:57,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 355881cf77b1bf9999fb7e0bd0dc8e2a: 2023-11-27 14:56:57,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1701096867430.355881cf77b1bf9999fb7e0bd0dc8e2a. 2023-11-27 14:56:57,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 39f113e3442d78447a6f24b4b73006c8, disabling compactions & flushes 2023-11-27 14:56:57,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:56:57,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:56:57,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. after waiting 0 ms 2023-11-27 14:56:57,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:56:57,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39f113e3442d78447a6f24b4b73006c8/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-11-27 14:56:57,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:56:57,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 39f113e3442d78447a6f24b4b73006c8: 2023-11-27 14:56:57,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1701096907129.39f113e3442d78447a6f24b4b73006c8. 2023-11-27 14:56:57,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3404a93afc957274c1f21854ed231972, disabling compactions & flushes 2023-11-27 14:56:57,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:57,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:57,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. after waiting 0 ms 2023-11-27 14:56:57,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:57,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/3404a93afc957274c1f21854ed231972/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-27 14:56:57,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:57,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3404a93afc957274c1f21854ed231972: 2023-11-27 14:56:57,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1701096970589.3404a93afc957274c1f21854ed231972. 2023-11-27 14:56:57,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f53154fc29f582902661898287c317b5, disabling compactions & flushes 2023-11-27 14:56:57,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:57,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:57,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. after waiting 0 ms 2023-11-27 14:56:57,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:57,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/f53154fc29f582902661898287c317b5/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-27 14:56:57,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:57,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f53154fc29f582902661898287c317b5: 2023-11-27 14:56:57,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1701096944559.f53154fc29f582902661898287c317b5. 2023-11-27 14:56:57,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e13a129f6f563cfb251e64b3f5d12815, disabling compactions & flushes 2023-11-27 14:56:57,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:56:57,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:56:57,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. after waiting 0 ms 2023-11-27 14:56:57,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:56:57,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/e13a129f6f563cfb251e64b3f5d12815/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-11-27 14:56:57,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:56:57,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e13a129f6f563cfb251e64b3f5d12815: 2023-11-27 14:56:57,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1701096891329.e13a129f6f563cfb251e64b3f5d12815. 2023-11-27 14:56:57,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 480bebe747409b1ccbbfa174b76957d7, disabling compactions & flushes 2023-11-27 14:56:57,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:57,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:57,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. after waiting 0 ms 2023-11-27 14:56:57,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:57,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/480bebe747409b1ccbbfa174b76957d7/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-27 14:56:57,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:57,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 480bebe747409b1ccbbfa174b76957d7: 2023-11-27 14:56:57,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1701096970589.480bebe747409b1ccbbfa174b76957d7. 2023-11-27 14:56:57,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 56c85fb3659c83b26d25644563d1ffc6, disabling compactions & flushes 2023-11-27 14:56:57,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:56:57,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:56:57,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. after waiting 0 ms 2023-11-27 14:56:57,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:56:57,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/56c85fb3659c83b26d25644563d1ffc6/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-11-27 14:56:57,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:56:57,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 56c85fb3659c83b26d25644563d1ffc6: 2023-11-27 14:56:57,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1701096891329.56c85fb3659c83b26d25644563d1ffc6. 2023-11-27 14:56:57,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1cf135bbee78d983093ed681ad9bc867, disabling compactions & flushes 2023-11-27 14:56:57,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:57,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:57,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. after waiting 0 ms 2023-11-27 14:56:57,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:57,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/ns/testMinorCompaction/1cf135bbee78d983093ed681ad9bc867/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-27 14:56:57,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:57,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1cf135bbee78d983093ed681ad9bc867: 2023-11-27 14:56:57,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1701096970589.1cf135bbee78d983093ed681ad9bc867. 2023-11-27 14:56:57,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b70da5c01d91e889a747889289d215f3, disabling compactions & flushes 2023-11-27 14:56:57,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:56:57,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:56:57,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. after waiting 0 ms 2023-11-27 14:56:57,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:56:57,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithWeeklyPolicy/b70da5c01d91e889a747889289d215f3/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-27 14:56:57,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:56:57,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b70da5c01d91e889a747889289d215f3: 2023-11-27 14:56:57,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1701096878344.b70da5c01d91e889a747889289d215f3. 2023-11-27 14:56:57,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 36cece40383490f5f6bb34f20558c5b9, disabling compactions & flushes 2023-11-27 14:56:57,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:56:57,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:56:57,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. after waiting 0 ms 2023-11-27 14:56:57,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:56:57,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/36cece40383490f5f6bb34f20558c5b9/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-11-27 14:56:57,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:56:57,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 36cece40383490f5f6bb34f20558c5b9: 2023-11-27 14:56:57,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1701096923841.36cece40383490f5f6bb34f20558c5b9. 2023-11-27 14:56:57,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 73f02d3ec84561cf82fbd52d7e1039a0, disabling compactions & flushes 2023-11-27 14:56:57,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:56:57,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:56:57,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. after waiting 0 ms 2023-11-27 14:56:57,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:56:57,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/73f02d3ec84561cf82fbd52d7e1039a0/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-11-27 14:56:57,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:56:57,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 73f02d3ec84561cf82fbd52d7e1039a0: 2023-11-27 14:56:57,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1701096853341.73f02d3ec84561cf82fbd52d7e1039a0. 2023-11-27 14:56:57,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cbfe88cf0c0ecdf504904655dc8d73d8, disabling compactions & flushes 2023-11-27 14:56:57,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:57,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:57,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. after waiting 0 ms 2023-11-27 14:56:57,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/cbfe88cf0c0ecdf504904655dc8d73d8/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-27 14:56:57,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cbfe88cf0c0ecdf504904655dc8d73d8: 2023-11-27 14:56:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1701097000422.cbfe88cf0c0ecdf504904655dc8d73d8. 2023-11-27 14:56:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d2db448818547f8516ae9f9e43b817b6, disabling compactions & flushes 2023-11-27 14:56:57,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. after waiting 0 ms 2023-11-27 14:56:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:57,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/d2db448818547f8516ae9f9e43b817b6/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-27 14:56:57,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:57,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d2db448818547f8516ae9f9e43b817b6: 2023-11-27 14:56:57,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1701096944559.d2db448818547f8516ae9f9e43b817b6. 2023-11-27 14:56:57,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 68bb86a7752ed0247b952754e90231e7, disabling compactions & flushes 2023-11-27 14:56:57,219 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:57,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:57,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. after waiting 0 ms 2023-11-27 14:56:57,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:57,230 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testCompactionWithHFileLink/68bb86a7752ed0247b952754e90231e7/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-27 14:56:57,231 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:57,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 68bb86a7752ed0247b952754e90231e7: 2023-11-27 14:56:57,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1701096944559.68bb86a7752ed0247b952754e90231e7. 2023-11-27 14:56:57,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d7e7b1527d4ce274f466396dcf7e3231, disabling compactions & flushes 2023-11-27 14:56:57,231 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:57,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:57,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. after waiting 0 ms 2023-11-27 14:56:57,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:57,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMinorCompactionWithMonthlyPolicy/d7e7b1527d4ce274f466396dcf7e3231/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-11-27 14:56:57,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:57,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d7e7b1527d4ce274f466396dcf7e3231: 2023-11-27 14:56:57,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1701096989345.d7e7b1527d4ce274f466396dcf7e3231. 2023-11-27 14:56:57,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d0e7e27ef1ca02c65babbede1720ba6c, disabling compactions & flushes 2023-11-27 14:56:57,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:57,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:57,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. after waiting 0 ms 2023-11-27 14:56:57,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:57,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/d0e7e27ef1ca02c65babbede1720ba6c/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-27 14:56:57,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:57,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d0e7e27ef1ca02c65babbede1720ba6c: 2023-11-27 14:56:57,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1701097000422.d0e7e27ef1ca02c65babbede1720ba6c. 2023-11-27 14:56:57,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4e630d9c8bdef52b2f9df3adf2c55277, disabling compactions & flushes 2023-11-27 14:56:57,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:56:57,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:56:57,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. after waiting 0 ms 2023-11-27 14:56:57,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:56:57,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testScannerOnBulkLoadRefHFiles/4e630d9c8bdef52b2f9df3adf2c55277/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-11-27 14:56:57,251 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:56:57,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4e630d9c8bdef52b2f9df3adf2c55277: 2023-11-27 14:56:57,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1701096891329.4e630d9c8bdef52b2f9df3adf2c55277. 2023-11-27 14:56:57,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8ae027c1211c228b679773750f57d6ba, disabling compactions & flushes 2023-11-27 14:56:57,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:56:57,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:56:57,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. after waiting 0 ms 2023-11-27 14:56:57,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:56:57,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionWithWeeklyPolicy/8ae027c1211c228b679773750f57d6ba/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-11-27 14:56:57,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:56:57,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8ae027c1211c228b679773750f57d6ba: 2023-11-27 14:56:57,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1701096894993.8ae027c1211c228b679773750f57d6ba. 2023-11-27 14:56:57,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f50f095dfe928b667178fa9ffe5f7373, disabling compactions & flushes 2023-11-27 14:56:57,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:57,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:57,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. after waiting 0 ms 2023-11-27 14:56:57,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:57,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/data/default/testMajorCompactionFromAdmin/f50f095dfe928b667178fa9ffe5f7373/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-27 14:56:57,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:57,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f50f095dfe928b667178fa9ffe5f7373: 2023-11-27 14:56:57,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1701097000422.f50f095dfe928b667178fa9ffe5f7373. 2023-11-27 14:56:57,322 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1173): stopping server jenkins-hbase4.apache.org,42669,1701096850360; all regions closed. 2023-11-27 14:56:57,331 DEBUG [RS:0;jenkins-hbase4:42669] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/oldWALs 2023-11-27 14:56:57,331 INFO [RS:0;jenkins-hbase4:42669] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42669%2C1701096850360.meta:.meta(num 1701096851896) 2023-11-27 14:56:57,336 DEBUG [RS:0;jenkins-hbase4:42669] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/oldWALs 2023-11-27 14:56:57,336 INFO [RS:0;jenkins-hbase4:42669] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42669%2C1701096850360:(num 1701096851804) 2023-11-27 14:56:57,336 DEBUG [RS:0;jenkins-hbase4:42669] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:57,336 INFO [RS:0;jenkins-hbase4:42669] regionserver.LeaseManager(133): Closed leases 2023-11-27 14:56:57,337 INFO [RS:0;jenkins-hbase4:42669] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-11-27 14:56:57,337 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-27 14:56:57,339 INFO [RS:0;jenkins-hbase4:42669] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42669 2023-11-27 14:56:57,350 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42669,1701096850360 2023-11-27 14:56:57,350 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-27 14:56:57,350 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-27 14:56:57,352 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42669,1701096850360] 2023-11-27 14:56:57,353 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42669,1701096850360; numProcessing=1 2023-11-27 14:56:57,354 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42669,1701096850360 already deleted, retry=false 2023-11-27 14:56:57,354 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase4.apache.org,42669,1701096850360 expired; onlineServers=0 2023-11-27 14:56:57,354 INFO [RegionServerTracker-0] regionserver.HRegionServer(2300): ***** STOPPING region server 'jenkins-hbase4.apache.org,39835,1701096849177' ***** 2023-11-27 14:56:57,354 INFO [RegionServerTracker-0] regionserver.HRegionServer(2314): STOPPED: Cluster shutdown set; onlineServer=0 2023-11-27 14:56:57,355 DEBUG [M:0;jenkins-hbase4:39835] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d09d80d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-27 14:56:57,355 INFO [M:0;jenkins-hbase4:39835] regionserver.HRegionServer(1147): stopping server jenkins-hbase4.apache.org,39835,1701096849177 2023-11-27 14:56:57,355 INFO [M:0;jenkins-hbase4:39835] regionserver.HRegionServer(1173): stopping server jenkins-hbase4.apache.org,39835,1701096849177; all regions closed. 2023-11-27 14:56:57,355 DEBUG [M:0;jenkins-hbase4:39835] ipc.AbstractRpcClient(489): Stopping rpc client 2023-11-27 14:56:57,355 DEBUG [M:0;jenkins-hbase4:39835] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-11-27 14:56:57,355 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-11-27 14:56:57,355 DEBUG [M:0;jenkins-hbase4:39835] cleaner.HFileCleaner(317): Stopping file delete threads 2023-11-27 14:56:57,355 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1701096851438] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1701096851438,5,FailOnTimeoutGroup] 2023-11-27 14:56:57,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1701096851438] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1701096851438,5,FailOnTimeoutGroup] 2023-11-27 14:56:57,356 INFO [M:0;jenkins-hbase4:39835] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-11-27 14:56:57,356 INFO [M:0;jenkins-hbase4:39835] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-11-27 14:56:57,356 INFO [M:0;jenkins-hbase4:39835] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-11-27 14:56:57,357 DEBUG [M:0;jenkins-hbase4:39835] master.HMaster(1512): Stopping service threads 2023-11-27 14:56:57,357 INFO [M:0;jenkins-hbase4:39835] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-11-27 14:56:57,357 ERROR [M:0;jenkins-hbase4:39835] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-19,5,PEWorkerGroup] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-11-27 14:56:57,358 INFO [M:0;jenkins-hbase4:39835] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-27 14:56:57,358 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-11-27 14:56:57,359 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-11-27 14:56:57,359 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-27 14:56:57,359 DEBUG [M:0;jenkins-hbase4:39835] zookeeper.ZKUtil(399): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-11-27 14:56:57,359 WARN [M:0;jenkins-hbase4:39835] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-11-27 14:56:57,359 INFO [M:0;jenkins-hbase4:39835] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-27 14:56:57,359 INFO [M:0;jenkins-hbase4:39835] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-11-27 14:56:57,360 DEBUG [M:0;jenkins-hbase4:39835] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-27 14:56:57,360 INFO [M:0;jenkins-hbase4:39835] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-27 14:56:57,360 DEBUG [M:0;jenkins-hbase4:39835] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-27 14:56:57,360 DEBUG [M:0;jenkins-hbase4:39835] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-27 14:56:57,360 DEBUG [M:0;jenkins-hbase4:39835] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-27 14:56:57,360 INFO [M:0;jenkins-hbase4:39835] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.97 KB heapSize=599.14 KB 2023-11-27 14:56:57,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-27 14:56:57,380 INFO [M:0;jenkins-hbase4:39835] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.97 KB at sequenceid=1011 (bloomFilter=true), to=hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3bf7b322a3084e059259bc77a54fcdcf 2023-11-27 14:56:57,387 INFO [M:0;jenkins-hbase4:39835] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3bf7b322a3084e059259bc77a54fcdcf 2023-11-27 14:56:57,388 DEBUG [M:0;jenkins-hbase4:39835] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3bf7b322a3084e059259bc77a54fcdcf as hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3bf7b322a3084e059259bc77a54fcdcf 2023-11-27 14:56:57,392 INFO [M:0;jenkins-hbase4:39835] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3bf7b322a3084e059259bc77a54fcdcf 2023-11-27 14:56:57,392 INFO [M:0;jenkins-hbase4:39835] regionserver.HStore(1080): Added hdfs://localhost:46277/user/jenkins/test-data/5cb92bda-7535-5b2c-1507-9d63ce4a46fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3bf7b322a3084e059259bc77a54fcdcf, entries=123, sequenceid=1011, filesize=33.3 K 2023-11-27 14:56:57,393 INFO [M:0;jenkins-hbase4:39835] regionserver.HRegion(2948): Finished flush of dataSize ~512.97 KB/525281, heapSize ~599.13 KB/613504, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 33ms, sequenceid=1011, compaction requested=false 2023-11-27 14:56:57,394 INFO [M:0;jenkins-hbase4:39835] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-27 14:56:57,394 DEBUG [M:0;jenkins-hbase4:39835] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-27 14:56:57,397 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-27 14:56:57,397 INFO [M:0;jenkins-hbase4:39835] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-11-27 14:56:57,398 INFO [M:0;jenkins-hbase4:39835] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39835 2023-11-27 14:56:57,401 DEBUG [M:0;jenkins-hbase4:39835] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,39835,1701096849177 already deleted, retry=false 2023-11-27 14:56:57,453 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-27 14:56:57,453 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): regionserver:42669-0x1002f55dfc60001, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-27 14:56:57,453 INFO [RS:0;jenkins-hbase4:42669] regionserver.HRegionServer(1230): Exiting; stopping=jenkins-hbase4.apache.org,42669,1701096850360; zookeeper connection closed. 2023-11-27 14:56:57,453 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@48555b10] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@48555b10 2023-11-27 14:56:57,453 INFO [Listener at localhost/37439] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-11-27 14:56:57,503 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-27 14:56:57,503 DEBUG [Listener at localhost/37439-EventThread] zookeeper.ZKWatcher(600): master:39835-0x1002f55dfc60000, quorum=127.0.0.1:61098, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-27 14:56:57,503 INFO [M:0;jenkins-hbase4:39835] regionserver.HRegionServer(1230): Exiting; stopping=jenkins-hbase4.apache.org,39835,1701096849177; zookeeper connection closed. 2023-11-27 14:56:57,505 WARN [Listener at localhost/37439] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-11-27 14:56:57,510 INFO [Listener at localhost/37439] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-27 14:56:57,617 WARN [BP-1195804122-172.31.14.131-1701096845386 heartbeating to localhost/127.0.0.1:46277] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-11-27 14:56:57,617 WARN [BP-1195804122-172.31.14.131-1701096845386 heartbeating to localhost/127.0.0.1:46277] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1195804122-172.31.14.131-1701096845386 (Datanode Uuid 31266597-0116-4265-9ab2-f25c98e274d7) service to localhost/127.0.0.1:46277 2023-11-27 14:56:57,619 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/cluster_e76484b8-e34a-cbd1-e5d0-706505e5be75/dfs/data/data1/current/BP-1195804122-172.31.14.131-1701096845386] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-27 14:56:57,619 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b80ad12d-8aa1-3b11-cd17-adba3bb7c30e/cluster_e76484b8-e34a-cbd1-e5d0-706505e5be75/dfs/data/data2/current/BP-1195804122-172.31.14.131-1701096845386] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-27 14:56:57,649 INFO [Listener at localhost/37439] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-27 14:56:57,782 INFO [Listener at localhost/37439] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-11-27 14:56:57,865 INFO [Listener at localhost/37439] hbase.HBaseTestingUtility(1293): Minicluster is down