2023-11-10 04:53:16,338 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-11-10 04:53:16,583 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec 2023-11-10 04:53:16,597 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-11-10 04:53:16,597 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/cluster_48cecc52-0985-53ae-674b-7482adaea2d3, deleteOnExit=true 2023-11-10 04:53:16,597 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-11-10 04:53:16,598 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/test.cache.data in system properties and HBase conf 2023-11-10 04:53:16,598 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/hadoop.tmp.dir in system properties and HBase conf 2023-11-10 04:53:16,599 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/hadoop.log.dir in system properties and HBase conf 2023-11-10 04:53:16,599 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/mapreduce.cluster.local.dir in system properties and HBase conf 2023-11-10 04:53:16,600 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-11-10 04:53:16,600 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-11-10 04:53:16,731 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-11-10 04:53:17,170 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-11-10 04:53:17,176 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-11-10 04:53:17,176 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-11-10 04:53:17,176 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-11-10 04:53:17,177 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-10 04:53:17,178 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-11-10 04:53:17,178 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-11-10 04:53:17,178 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-10 04:53:17,179 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-10 04:53:17,179 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-11-10 04:53:17,180 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/nfs.dump.dir in system properties and HBase conf 2023-11-10 04:53:17,180 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/java.io.tmpdir in system properties and HBase conf 2023-11-10 04:53:17,181 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-10 04:53:17,181 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-11-10 04:53:17,182 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-11-10 04:53:17,799 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-10 04:53:17,804 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-10 04:53:18,096 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-11-10 04:53:18,272 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-11-10 04:53:18,294 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-10 04:53:18,334 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-11-10 04:53:18,368 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/java.io.tmpdir/Jetty_localhost_40405_hdfs____.aop4bd/webapp 2023-11-10 04:53:18,504 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40405 2023-11-10 04:53:18,515 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-10 04:53:18,515 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-10 04:53:19,059 WARN [Listener at localhost/43735] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-10 04:53:19,141 WARN [Listener at localhost/43735] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-11-10 04:53:19,164 WARN [Listener at localhost/43735] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-10 04:53:19,171 INFO [Listener at localhost/43735] log.Slf4jLog(67): jetty-6.1.26 2023-11-10 04:53:19,176 INFO [Listener at localhost/43735] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/java.io.tmpdir/Jetty_localhost_35677_datanode____5n2ztn/webapp 2023-11-10 04:53:19,275 INFO [Listener at localhost/43735] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35677 2023-11-10 04:53:19,713 WARN [Listener at localhost/35765] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-10 04:53:20,253 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd5d91aaf77800868: Processing first storage report for DS-3a8b3522-63a3-49ca-842f-75609aa709e5 from datanode ef526ac7-835b-4e32-8089-9ce0eb46f62e 2023-11-10 04:53:20,255 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd5d91aaf77800868: from storage DS-3a8b3522-63a3-49ca-842f-75609aa709e5 node DatanodeRegistration(127.0.0.1:35053, datanodeUuid=ef526ac7-835b-4e32-8089-9ce0eb46f62e, infoPort=44663, infoSecurePort=0, ipcPort=35765, storageInfo=lv=-57;cid=testClusterID;nsid=419574904;c=1699591997884), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-11-10 04:53:20,255 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd5d91aaf77800868: Processing first storage report for DS-f4bbc258-073b-4f19-900e-22d6bc9372f2 from datanode ef526ac7-835b-4e32-8089-9ce0eb46f62e 2023-11-10 04:53:20,255 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd5d91aaf77800868: from storage DS-f4bbc258-073b-4f19-900e-22d6bc9372f2 node DatanodeRegistration(127.0.0.1:35053, datanodeUuid=ef526ac7-835b-4e32-8089-9ce0eb46f62e, infoPort=44663, infoSecurePort=0, ipcPort=35765, storageInfo=lv=-57;cid=testClusterID;nsid=419574904;c=1699591997884), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-11-10 04:53:20,340 DEBUG [Listener at localhost/35765] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec 2023-11-10 04:53:20,453 INFO [Listener at localhost/35765] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/cluster_48cecc52-0985-53ae-674b-7482adaea2d3/zookeeper_0, clientPort=51169, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/cluster_48cecc52-0985-53ae-674b-7482adaea2d3/zookeeper_0/version-2, dataDirSize=457 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/cluster_48cecc52-0985-53ae-674b-7482adaea2d3/zookeeper_0/version-2, dataLogSize=457 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, clientPortListenBacklog=-1, serverId=0 2023-11-10 04:53:20,467 INFO [Listener at localhost/35765] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51169 2023-11-10 04:53:20,474 INFO [Listener at localhost/35765] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-10 04:53:20,476 INFO [Listener at localhost/35765] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-10 04:53:21,177 INFO [Listener at localhost/35765] util.FSUtils(471): Created version file at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8 with version=8 2023-11-10 04:53:21,177 INFO [Listener at localhost/35765] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/hbase-staging 2023-11-10 04:53:21,471 INFO [Listener at localhost/35765] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-11-10 04:53:21,924 INFO [Listener at localhost/35765] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-11-10 04:53:21,955 INFO [Listener at localhost/35765] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-10 04:53:21,956 INFO [Listener at localhost/35765] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-10 04:53:21,956 INFO [Listener at localhost/35765] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-10 04:53:21,956 INFO [Listener at localhost/35765] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-10 04:53:21,956 INFO [Listener at localhost/35765] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-10 04:53:22,109 INFO [Listener at localhost/35765] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-11-10 04:53:22,191 DEBUG [Listener at localhost/35765] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-11-10 04:53:22,285 INFO [Listener at localhost/35765] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46447 2023-11-10 04:53:22,295 INFO [Listener at localhost/35765] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-10 04:53:22,297 INFO [Listener at localhost/35765] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-10 04:53:22,319 INFO [Listener at localhost/35765] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46447 connecting to ZooKeeper ensemble=127.0.0.1:51169 2023-11-10 04:53:22,362 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:464470x0, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-10 04:53:22,364 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:46447-0x101ce530b520000 connected 2023-11-10 04:53:22,396 DEBUG [Listener at localhost/35765] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-10 04:53:22,397 DEBUG [Listener at localhost/35765] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-10 04:53:22,400 DEBUG [Listener at localhost/35765] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-10 04:53:22,408 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46447 2023-11-10 04:53:22,408 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46447 2023-11-10 04:53:22,408 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46447 2023-11-10 04:53:22,409 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46447 2023-11-10 04:53:22,410 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46447 2023-11-10 04:53:22,415 INFO [Listener at localhost/35765] master.HMaster(444): hbase.rootdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8, hbase.cluster.distributed=false 2023-11-10 04:53:22,486 INFO [Listener at localhost/35765] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-11-10 04:53:22,486 INFO [Listener at localhost/35765] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-10 04:53:22,486 INFO [Listener at localhost/35765] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-10 04:53:22,486 INFO [Listener at localhost/35765] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-10 04:53:22,486 INFO [Listener at localhost/35765] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-10 04:53:22,486 INFO [Listener at localhost/35765] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-10 04:53:22,491 INFO [Listener at localhost/35765] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-11-10 04:53:22,494 INFO [Listener at localhost/35765] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36419 2023-11-10 04:53:22,496 INFO [Listener at localhost/35765] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-11-10 04:53:22,503 DEBUG [Listener at localhost/35765] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-11-10 04:53:22,504 INFO [Listener at localhost/35765] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-10 04:53:22,506 INFO [Listener at localhost/35765] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-10 04:53:22,508 INFO [Listener at localhost/35765] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36419 connecting to ZooKeeper ensemble=127.0.0.1:51169 2023-11-10 04:53:22,512 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:364190x0, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-10 04:53:22,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36419-0x101ce530b520001 connected 2023-11-10 04:53:22,513 DEBUG [Listener at localhost/35765] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-10 04:53:22,515 DEBUG [Listener at localhost/35765] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-10 04:53:22,515 DEBUG [Listener at localhost/35765] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-10 04:53:22,516 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36419 2023-11-10 04:53:22,516 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36419 2023-11-10 04:53:22,516 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36419 2023-11-10 04:53:22,520 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36419 2023-11-10 04:53:22,520 DEBUG [Listener at localhost/35765] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36419 2023-11-10 04:53:22,522 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:53:22,532 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-10 04:53:22,534 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:53:22,553 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-10 04:53:22,553 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-10 04:53:22,553 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:53:22,554 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-10 04:53:22,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,46447,1699592001315 from backup master directory 2023-11-10 04:53:22,555 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-10 04:53:22,559 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:53:22,559 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-10 04:53:22,559 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-10 04:53:22,560 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:53:22,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-11-10 04:53:22,563 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-11-10 04:53:23,049 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/hbase.id with ID: f41a00ec-b900-4a67-833e-1be9521f67ba 2023-11-10 04:53:23,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-10 04:53:23,106 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:53:23,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x295edb9d to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:53:23,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@240a4281, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:53:23,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-11-10 04:53:23,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-11-10 04:53:23,229 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-11-10 04:53:23,229 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-11-10 04:53:23,231 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-10 04:53:23,235 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-10 04:53:23,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-10 04:53:23,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/data/master/store-tmp 2023-11-10 04:53:23,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:23,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-10 04:53:23,698 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-10 04:53:23,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-10 04:53:23,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-10 04:53:23,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-10 04:53:23,699 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-10 04:53:23,699 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-10 04:53:23,700 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/WALs/jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:53:23,720 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46447%2C1699592001315, suffix=, logDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/WALs/jenkins-hbase4.apache.org,46447,1699592001315, archiveDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/oldWALs, maxLogs=10 2023-11-10 04:53:23,784 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35053,DS-3a8b3522-63a3-49ca-842f-75609aa709e5,DISK] 2023-11-10 04:53:23,795 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-11-10 04:53:23,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/WALs/jenkins-hbase4.apache.org,46447,1699592001315/jenkins-hbase4.apache.org%2C46447%2C1699592001315.1699592003730 2023-11-10 04:53:23,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35053,DS-3a8b3522-63a3-49ca-842f-75609aa709e5,DISK]] 2023-11-10 04:53:23,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:53:23,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:23,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-11-10 04:53:23,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-11-10 04:53:23,968 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-11-10 04:53:23,978 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-11-10 04:53:24,008 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-11-10 04:53:24,023 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:24,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-10 04:53:24,033 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-10 04:53:24,052 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-11-10 04:53:24,057 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:53:24,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67255967, jitterRate=0.0021920055150985718}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:53:24,060 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-10 04:53:24,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-11-10 04:53:24,087 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-11-10 04:53:24,088 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-11-10 04:53:24,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-10 04:53:24,093 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 1 msec 2023-11-10 04:53:24,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 35 msec 2023-11-10 04:53:24,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-11-10 04:53:24,160 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-11-10 04:53:24,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-11-10 04:53:24,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-11-10 04:53:24,200 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-11-10 04:53:24,202 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-11-10 04:53:24,208 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-11-10 04:53:24,212 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-11-10 04:53:24,218 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:53:24,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-11-10 04:53:24,220 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-11-10 04:53:24,238 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-11-10 04:53:24,244 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-10 04:53:24,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:53:24,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-10 04:53:24,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,46447,1699592001315, sessionid=0x101ce530b520000, setting cluster-up flag (Was=false) 2023-11-10 04:53:24,267 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:53:24,276 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-11-10 04:53:24,277 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:53:24,283 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:53:24,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-11-10 04:53:24,292 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:53:24,295 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(304): Couldn't delete working snapshot directory: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp 2023-11-10 04:53:24,326 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(951): ClusterId : f41a00ec-b900-4a67-833e-1be9521f67ba 2023-11-10 04:53:24,331 DEBUG [RS:0;jenkins-hbase4:36419] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-11-10 04:53:24,338 DEBUG [RS:0;jenkins-hbase4:36419] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-11-10 04:53:24,338 DEBUG [RS:0;jenkins-hbase4:36419] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-11-10 04:53:24,342 DEBUG [RS:0;jenkins-hbase4:36419] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-11-10 04:53:24,343 DEBUG [RS:0;jenkins-hbase4:36419] zookeeper.ReadOnlyZKClient(139): Connect 0x04ff47a3 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:53:24,349 DEBUG [RS:0;jenkins-hbase4:36419] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a38b94c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:53:24,350 DEBUG [RS:0;jenkins-hbase4:36419] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e072cd3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-10 04:53:24,380 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36419 2023-11-10 04:53:24,385 INFO [RS:0;jenkins-hbase4:36419] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-11-10 04:53:24,385 INFO [RS:0;jenkins-hbase4:36419] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-11-10 04:53:24,386 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1022): About to register with Master. 2023-11-10 04:53:24,390 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,46447,1699592001315 with isa=jenkins-hbase4.apache.org/172.31.14.131:36419, startcode=1699592002485 2023-11-10 04:53:24,415 DEBUG [RS:0;jenkins-hbase4:36419] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-11-10 04:53:24,450 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-11-10 04:53:24,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-10 04:53:24,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-10 04:53:24,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-10 04:53:24,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-10 04:53:24,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-11-10 04:53:24,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-10 04:53:24,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,464 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1699592034464 2023-11-10 04:53:24,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-11-10 04:53:24,468 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-11-10 04:53:24,469 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-11-10 04:53:24,474 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-10 04:53:24,475 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55359, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-11-10 04:53:24,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-11-10 04:53:24,483 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46447] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-11-10 04:53:24,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-11-10 04:53:24,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-11-10 04:53:24,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-11-10 04:53:24,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-11-10 04:53:24,487 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-11-10 04:53:24,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-11-10 04:53:24,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-11-10 04:53:24,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-11-10 04:53:24,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-11-10 04:53:24,495 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1699592004495,5,FailOnTimeoutGroup] 2023-11-10 04:53:24,496 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1699592004496,5,FailOnTimeoutGroup] 2023-11-10 04:53:24,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-11-10 04:53:24,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,516 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(2830): Master is not running yet 2023-11-10 04:53:24,517 WARN [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-11-10 04:53:24,618 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,46447,1699592001315 with isa=jenkins-hbase4.apache.org/172.31.14.131:36419, startcode=1699592002485 2023-11-10 04:53:24,624 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46447] master.ServerManager(388): Registering regionserver=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:24,629 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8 2023-11-10 04:53:24,629 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:43735 2023-11-10 04:53:24,629 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-11-10 04:53:24,635 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-10 04:53:24,635 DEBUG [RS:0;jenkins-hbase4:36419] zookeeper.ZKUtil(162): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:24,635 WARN [RS:0;jenkins-hbase4:36419] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-10 04:53:24,636 INFO [RS:0;jenkins-hbase4:36419] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-10 04:53:24,636 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1948): logDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/WALs/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:24,637 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36419,1699592002485] 2023-11-10 04:53:24,644 DEBUG [RS:0;jenkins-hbase4:36419] zookeeper.ZKUtil(162): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:24,654 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-11-10 04:53:24,662 INFO [RS:0;jenkins-hbase4:36419] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-11-10 04:53:24,681 INFO [RS:0;jenkins-hbase4:36419] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-11-10 04:53:24,684 INFO [RS:0;jenkins-hbase4:36419] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:53:24,684 INFO [RS:0;jenkins-hbase4:36419] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,685 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-11-10 04:53:24,693 INFO [RS:0;jenkins-hbase4:36419] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,693 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,695 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,695 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,695 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,695 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,695 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-10 04:53:24,696 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,696 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,696 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,696 DEBUG [RS:0;jenkins-hbase4:36419] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-10 04:53:24,697 INFO [RS:0;jenkins-hbase4:36419] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,697 INFO [RS:0;jenkins-hbase4:36419] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,697 INFO [RS:0;jenkins-hbase4:36419] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,715 INFO [RS:0;jenkins-hbase4:36419] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-11-10 04:53:24,718 INFO [RS:0;jenkins-hbase4:36419] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36419,1699592002485-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:24,735 INFO [RS:0;jenkins-hbase4:36419] regionserver.Replication(203): jenkins-hbase4.apache.org,36419,1699592002485 started 2023-11-10 04:53:24,735 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36419,1699592002485, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36419, sessionid=0x101ce530b520001 2023-11-10 04:53:24,735 DEBUG [RS:0;jenkins-hbase4:36419] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-11-10 04:53:24,735 DEBUG [RS:0;jenkins-hbase4:36419] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:24,735 DEBUG [RS:0;jenkins-hbase4:36419] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36419,1699592002485' 2023-11-10 04:53:24,735 DEBUG [RS:0;jenkins-hbase4:36419] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:24,736 DEBUG [RS:0;jenkins-hbase4:36419] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:24,737 DEBUG [RS:0;jenkins-hbase4:36419] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-11-10 04:53:24,737 DEBUG [RS:0;jenkins-hbase4:36419] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-11-10 04:53:24,737 DEBUG [RS:0;jenkins-hbase4:36419] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:24,737 DEBUG [RS:0;jenkins-hbase4:36419] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36419,1699592002485' 2023-11-10 04:53:24,737 DEBUG [RS:0;jenkins-hbase4:36419] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-10 04:53:24,738 DEBUG [RS:0;jenkins-hbase4:36419] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-10 04:53:24,739 DEBUG [RS:0;jenkins-hbase4:36419] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-11-10 04:53:24,739 INFO [RS:0;jenkins-hbase4:36419] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-11-10 04:53:24,739 INFO [RS:0;jenkins-hbase4:36419] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-11-10 04:53:24,850 INFO [RS:0;jenkins-hbase4:36419] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36419%2C1699592002485, suffix=, logDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/WALs/jenkins-hbase4.apache.org,36419,1699592002485, archiveDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/oldWALs, maxLogs=32 2023-11-10 04:53:24,869 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35053,DS-3a8b3522-63a3-49ca-842f-75609aa709e5,DISK] 2023-11-10 04:53:24,873 INFO [RS:0;jenkins-hbase4:36419] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/WALs/jenkins-hbase4.apache.org,36419,1699592002485/jenkins-hbase4.apache.org%2C36419%2C1699592002485.1699592004853 2023-11-10 04:53:24,873 DEBUG [RS:0;jenkins-hbase4:36419] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35053,DS-3a8b3522-63a3-49ca-842f-75609aa709e5,DISK]] 2023-11-10 04:53:24,914 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-10 04:53:24,915 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-10 04:53:24,915 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8 2023-11-10 04:53:25,333 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:25,336 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-10 04:53:25,339 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/info 2023-11-10 04:53:25,340 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-10 04:53:25,341 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:25,341 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-10 04:53:25,344 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/rep_barrier 2023-11-10 04:53:25,345 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-10 04:53:25,346 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:25,346 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-10 04:53:25,348 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/table 2023-11-10 04:53:25,349 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-10 04:53:25,349 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:25,351 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740 2023-11-10 04:53:25,352 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740 2023-11-10 04:53:25,356 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-10 04:53:25,358 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-10 04:53:25,361 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:53:25,362 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65749577, jitterRate=-0.0202549546957016}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-10 04:53:25,362 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-10 04:53:25,362 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-10 04:53:25,362 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-10 04:53:25,362 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-10 04:53:25,362 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-10 04:53:25,363 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-10 04:53:25,363 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-10 04:53:25,364 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-10 04:53:25,369 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-11-10 04:53:25,369 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-11-10 04:53:25,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-11-10 04:53:25,387 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-11-10 04:53:25,389 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-11-10 04:53:25,541 DEBUG [jenkins-hbase4:46447] assignment.AssignmentManager(2186): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-11-10 04:53:25,544 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36419,1699592002485, state=OPENING 2023-11-10 04:53:25,553 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-11-10 04:53:25,555 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:53:25,556 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-10 04:53:25,558 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:53:25,740 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:25,743 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-10 04:53:25,747 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43202, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-10 04:53:25,758 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-11-10 04:53:25,758 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-10 04:53:25,762 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36419%2C1699592002485.meta, suffix=.meta, logDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/WALs/jenkins-hbase4.apache.org,36419,1699592002485, archiveDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/oldWALs, maxLogs=32 2023-11-10 04:53:25,780 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35053,DS-3a8b3522-63a3-49ca-842f-75609aa709e5,DISK] 2023-11-10 04:53:25,784 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/WALs/jenkins-hbase4.apache.org,36419,1699592002485/jenkins-hbase4.apache.org%2C36419%2C1699592002485.meta.1699592005764.meta 2023-11-10 04:53:25,784 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35053,DS-3a8b3522-63a3-49ca-842f-75609aa709e5,DISK]] 2023-11-10 04:53:25,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:53:25,787 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-11-10 04:53:25,807 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-11-10 04:53:25,814 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-11-10 04:53:25,821 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-11-10 04:53:25,821 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:25,821 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-11-10 04:53:25,821 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-11-10 04:53:25,825 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-10 04:53:25,826 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/info 2023-11-10 04:53:25,827 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/info 2023-11-10 04:53:25,827 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-10 04:53:25,828 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:25,829 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-10 04:53:25,830 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/rep_barrier 2023-11-10 04:53:25,830 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/rep_barrier 2023-11-10 04:53:25,831 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-10 04:53:25,832 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:25,832 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-10 04:53:25,834 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/table 2023-11-10 04:53:25,834 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/table 2023-11-10 04:53:25,835 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-10 04:53:25,836 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:25,837 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740 2023-11-10 04:53:25,846 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740 2023-11-10 04:53:25,851 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-10 04:53:25,853 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-10 04:53:25,855 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66118017, jitterRate=-0.014764770865440369}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-10 04:53:25,855 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-10 04:53:25,866 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1699592005733 2023-11-10 04:53:25,884 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-11-10 04:53:25,885 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-11-10 04:53:25,885 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36419,1699592002485, state=OPEN 2023-11-10 04:53:25,892 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-11-10 04:53:25,892 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-10 04:53:25,896 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-11-10 04:53:25,896 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36419,1699592002485 in 334 msec 2023-11-10 04:53:25,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-11-10 04:53:25,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 521 msec 2023-11-10 04:53:25,907 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5440 sec 2023-11-10 04:53:25,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1699592005907, completionTime=-1 2023-11-10 04:53:25,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(808): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-11-10 04:53:25,907 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1527): Joining cluster... 2023-11-10 04:53:25,968 DEBUG [hconnection-0x537b9561-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:53:25,972 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43204, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:53:25,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1539): Number of RegionServers=1 2023-11-10 04:53:25,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1699592065989 2023-11-10 04:53:25,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1699592125989 2023-11-10 04:53:25,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1546): Joined the cluster in 82 msec 2023-11-10 04:53:26,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46447,1699592001315-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:26,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46447,1699592001315-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:26,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46447,1699592001315-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:26,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:46447, period=300000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:26,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-11-10 04:53:26,019 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-11-10 04:53:26,027 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-11-10 04:53:26,028 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-10 04:53:26,038 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-11-10 04:53:26,040 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:53:26,042 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:53:26,064 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:53:26,067 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853 empty. 2023-11-10 04:53:26,067 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:53:26,067 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-11-10 04:53:26,089 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-11-10 04:53:26,091 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 49d033c6fa4701f6eda4cca344a97853, NAME => 'hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:53:26,502 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:26,502 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 49d033c6fa4701f6eda4cca344a97853, disabling compactions & flushes 2023-11-10 04:53:26,502 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:53:26,502 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:53:26,502 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. after waiting 0 ms 2023-11-10 04:53:26,502 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:53:26,502 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:53:26,502 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 49d033c6fa4701f6eda4cca344a97853: 2023-11-10 04:53:26,508 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:53:26,526 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1699592006511"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592006511"}]},"ts":"1699592006511"} 2023-11-10 04:53:26,557 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-10 04:53:26,559 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:53:26,564 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592006560"}]},"ts":"1699592006560"} 2023-11-10 04:53:26,571 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-11-10 04:53:26,581 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=49d033c6fa4701f6eda4cca344a97853, ASSIGN}] 2023-11-10 04:53:26,584 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=49d033c6fa4701f6eda4cca344a97853, ASSIGN 2023-11-10 04:53:26,587 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=49d033c6fa4701f6eda4cca344a97853, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:53:26,738 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=49d033c6fa4701f6eda4cca344a97853, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:26,739 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1699592006738"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592006738"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592006738"}]},"ts":"1699592006738"} 2023-11-10 04:53:26,745 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 49d033c6fa4701f6eda4cca344a97853, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:53:26,899 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:26,906 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:53:26,907 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 49d033c6fa4701f6eda4cca344a97853, NAME => 'hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:53:26,908 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:53:26,908 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:26,908 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:53:26,908 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:53:26,910 INFO [StoreOpener-49d033c6fa4701f6eda4cca344a97853-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:53:26,912 DEBUG [StoreOpener-49d033c6fa4701f6eda4cca344a97853-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853/info 2023-11-10 04:53:26,912 DEBUG [StoreOpener-49d033c6fa4701f6eda4cca344a97853-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853/info 2023-11-10 04:53:26,913 INFO [StoreOpener-49d033c6fa4701f6eda4cca344a97853-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 49d033c6fa4701f6eda4cca344a97853 columnFamilyName info 2023-11-10 04:53:26,914 INFO [StoreOpener-49d033c6fa4701f6eda4cca344a97853-1] regionserver.HStore(310): Store=49d033c6fa4701f6eda4cca344a97853/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:26,915 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:53:26,916 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:53:26,921 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:53:26,924 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:53:26,925 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 49d033c6fa4701f6eda4cca344a97853; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70397553, jitterRate=0.049005284905433655}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:53:26,925 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 49d033c6fa4701f6eda4cca344a97853: 2023-11-10 04:53:26,927 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853., pid=6, masterSystemTime=1699592006899 2023-11-10 04:53:26,931 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:53:26,931 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:53:26,932 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=49d033c6fa4701f6eda4cca344a97853, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:26,933 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1699592006932"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592006932"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592006932"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592006932"}]},"ts":"1699592006932"} 2023-11-10 04:53:26,940 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-11-10 04:53:26,940 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 49d033c6fa4701f6eda4cca344a97853, server=jenkins-hbase4.apache.org,36419,1699592002485 in 191 msec 2023-11-10 04:53:26,943 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-11-10 04:53:26,943 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=49d033c6fa4701f6eda4cca344a97853, ASSIGN in 359 msec 2023-11-10 04:53:26,944 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:53:26,945 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592006945"}]},"ts":"1699592006945"} 2023-11-10 04:53:26,947 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-11-10 04:53:26,951 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:53:26,953 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 922 msec 2023-11-10 04:53:27,041 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-11-10 04:53:27,043 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-11-10 04:53:27,043 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:53:27,085 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-11-10 04:53:27,100 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-10 04:53:27,105 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-11-10 04:53:27,108 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-11-10 04:53:27,119 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-10 04:53:27,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 14 msec 2023-11-10 04:53:27,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-11-10 04:53:27,139 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-11-10 04:53:27,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.579sec 2023-11-10 04:53:27,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-11-10 04:53:27,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-11-10 04:53:27,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-11-10 04:53:27,143 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46447,1699592001315-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-11-10 04:53:27,143 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46447,1699592001315-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-11-10 04:53:27,153 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-11-10 04:53:27,237 DEBUG [Listener at localhost/35765] zookeeper.ReadOnlyZKClient(139): Connect 0x2324da13 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:53:27,241 DEBUG [Listener at localhost/35765] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@438af03d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:53:27,254 DEBUG [hconnection-0x296ad028-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:53:27,265 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43214, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:53:27,275 INFO [Listener at localhost/35765] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:53:27,277 DEBUG [Listener at localhost/35765] zookeeper.ReadOnlyZKClient(139): Connect 0x05ae2dad to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:53:27,281 DEBUG [Listener at localhost/35765] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e8f0a32, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:53:27,282 INFO [Listener at localhost/35765] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-10 04:53:27,341 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=214, OpenFileDescriptor=505, MaxFileDescriptor=60000, SystemLoadAverage=330, ProcessCount=170, AvailableMemoryMB=11485 2023-11-10 04:53:27,352 DEBUG [Listener at localhost/35765] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:53:27,355 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38618, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:53:27,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-10 04:53:27,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:27,371 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:53:27,373 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:53:27,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-11-10 04:53:27,376 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:27,378 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e empty. 2023-11-10 04:53:27,379 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:27,379 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-11-10 04:53:27,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-10 04:53:27,395 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-11-10 04:53:27,397 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => d0cb33d1bbec7f17d50fcde1f31e585e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:53:27,408 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:27,409 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing d0cb33d1bbec7f17d50fcde1f31e585e, disabling compactions & flushes 2023-11-10 04:53:27,409 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:27,409 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:27,409 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. after waiting 0 ms 2023-11-10 04:53:27,409 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:27,409 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:27,409 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:27,412 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:53:27,414 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1699592007414"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592007414"}]},"ts":"1699592007414"} 2023-11-10 04:53:27,417 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-10 04:53:27,418 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:53:27,418 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592007418"}]},"ts":"1699592007418"} 2023-11-10 04:53:27,420 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-11-10 04:53:27,425 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=d0cb33d1bbec7f17d50fcde1f31e585e, ASSIGN}] 2023-11-10 04:53:27,427 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=d0cb33d1bbec7f17d50fcde1f31e585e, ASSIGN 2023-11-10 04:53:27,429 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=d0cb33d1bbec7f17d50fcde1f31e585e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:53:27,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-10 04:53:27,580 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d0cb33d1bbec7f17d50fcde1f31e585e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:27,580 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1699592007580"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592007580"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592007580"}]},"ts":"1699592007580"} 2023-11-10 04:53:27,583 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure d0cb33d1bbec7f17d50fcde1f31e585e, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:53:27,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-10 04:53:27,737 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:27,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:27,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d0cb33d1bbec7f17d50fcde1f31e585e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:53:27,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:27,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:27,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:27,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:27,747 INFO [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:27,750 DEBUG [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:27,750 DEBUG [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:27,754 INFO [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0cb33d1bbec7f17d50fcde1f31e585e columnFamilyName family1 2023-11-10 04:53:27,757 INFO [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] regionserver.HStore(310): Store=d0cb33d1bbec7f17d50fcde1f31e585e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:27,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:27,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:27,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:27,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:53:27,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d0cb33d1bbec7f17d50fcde1f31e585e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70325157, jitterRate=0.04792650043964386}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:53:27,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:27,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e., pid=11, masterSystemTime=1699592007736 2023-11-10 04:53:27,790 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d0cb33d1bbec7f17d50fcde1f31e585e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:27,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:27,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:27,790 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1699592007789"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592007789"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592007789"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592007789"}]},"ts":"1699592007789"} 2023-11-10 04:53:27,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-11-10 04:53:27,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure d0cb33d1bbec7f17d50fcde1f31e585e, server=jenkins-hbase4.apache.org,36419,1699592002485 in 210 msec 2023-11-10 04:53:27,802 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-11-10 04:53:27,802 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=d0cb33d1bbec7f17d50fcde1f31e585e, ASSIGN in 373 msec 2023-11-10 04:53:27,803 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:53:27,804 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592007803"}]},"ts":"1699592007803"} 2023-11-10 04:53:27,806 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-11-10 04:53:27,810 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:53:27,812 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 443 msec 2023-11-10 04:53:27,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-10 04:53:27,995 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-11-10 04:53:28,013 DEBUG [hconnection-0x7e3e3678-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:53:28,017 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43228, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:53:28,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:28,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,172 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:28,172 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:28,173 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:28,173 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:28,174 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,174 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,176 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:28,176 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:28,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:28,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,178 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,179 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:28,179 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:28,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,182 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,182 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:28,182 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:28,183 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:28,184 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:28,184 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:28,184 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:28,185 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:28,186 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-10 04:53:28,343 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a1c3eeca78954369bbcf502557a298e5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a1c3eeca78954369bbcf502557a298e5 2023-11-10 04:53:28,759 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/1076a9f70e4840f3a862e8c3cbea0982 2023-11-10 04:53:28,772 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/1076a9f70e4840f3a862e8c3cbea0982 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1076a9f70e4840f3a862e8c3cbea0982 2023-11-10 04:53:28,784 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1076a9f70e4840f3a862e8c3cbea0982, entries=1000, sequenceid=5, filesize=175.8 K 2023-11-10 04:53:28,792 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 605ms, sequenceid=5, compaction requested=false 2023-11-10 04:53:28,794 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:28,797 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:28,797 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:28,797 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:28,797 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:28,797 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,798 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:28,798 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:28,800 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:28,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:28,801 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,801 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:28,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:28,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:28,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:28,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:28,804 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:28,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24982b55[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:28,804 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,807 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,807 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:28,807 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:28,807 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:28,808 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,808 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:28,810 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:28,810 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,811 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:28,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:28,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:28,811 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:28,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:28,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:28,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:28,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:28,815 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:28,815 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:28,815 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:28,816 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:28,816 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,816 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:28,819 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,819 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:28,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:28,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,819 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:28,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:28,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:28,820 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:28,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:28,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,823 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:28,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,833 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,833 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,833 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,833 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:28,833 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:28,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:28,833 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,834 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:28,834 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,834 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,834 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:28,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:28,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:28,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-10 04:53:28,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:28,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:28,837 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:28,837 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:28,937 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:28,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:28,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:28,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:28,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,960 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:28,960 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:28,961 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:28,961 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:28,962 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,962 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,967 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,967 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:28,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:28,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:28,968 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:28,968 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:28,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:28,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,969 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-10 04:53:28,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:28,969 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:28,970 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:28,970 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:28,970 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:28,970 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:28,970 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:28,970 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:28,970 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-10 04:53:29,398 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ded5070166b1476e8aa4d915b128c68a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ded5070166b1476e8aa4d915b128c68a 2023-11-10 04:53:29,413 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/83bc815d2cea41ac833e7e03344aa3af 2023-11-10 04:53:29,423 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/83bc815d2cea41ac833e7e03344aa3af as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/83bc815d2cea41ac833e7e03344aa3af 2023-11-10 04:53:29,432 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/83bc815d2cea41ac833e7e03344aa3af, entries=1, sequenceid=9, filesize=5.0 K 2023-11-10 04:53:29,433 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 463ms, sequenceid=9, compaction requested=false 2023-11-10 04:53:29,433 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:29,434 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:29,434 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:29,434 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:29,434 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,434 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:29,434 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:29,438 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:29,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,438 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:29,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:29,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:29,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:29,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:29,441 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f5f0253[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:29,441 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:29,441 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,443 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,443 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:29,443 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:29,443 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:29,443 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,443 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:29,445 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:29,445 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,445 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:29,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:29,445 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:29,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:29,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:29,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:29,450 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:29,450 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:29,450 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:29,450 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:29,450 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,450 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:29,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:29,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:29,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:29,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:29,456 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:29,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,458 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:29,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:29,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:29,463 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,463 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:29,463 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:29,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:29,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:29,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:29,463 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,463 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,463 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:29,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:29,464 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:29,464 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:29,564 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:29,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:29,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:29,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:29,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,575 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:29,576 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:29,576 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:29,576 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:29,577 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,577 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,578 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:29,578 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:29,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:29,579 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:29,579 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:29,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,580 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:29,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,580 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,581 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-10 04:53:29,581 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:29,581 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:29,581 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:29,581 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:29,582 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:29,582 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:29,582 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:29,582 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:29,582 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:29,623 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201fdd9ba7d50e8451fb9b94634a04e8692 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201fdd9ba7d50e8451fb9b94634a04e8692 2023-11-10 04:53:30,042 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/0bcbd2889e1a480c82257f5241c33448 2023-11-10 04:53:30,053 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/0bcbd2889e1a480c82257f5241c33448 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bcbd2889e1a480c82257f5241c33448 2023-11-10 04:53:30,061 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bcbd2889e1a480c82257f5241c33448, entries=1, sequenceid=13, filesize=5.0 K 2023-11-10 04:53:30,062 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 480ms, sequenceid=13, compaction requested=true 2023-11-10 04:53:30,062 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:30,062 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:30,063 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:30,063 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:30,063 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,063 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:30,063 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:30,066 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,066 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:30,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:30,066 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,067 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:30,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:30,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:30,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:30,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:30,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@168d8f22[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:30,069 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:30,069 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,072 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,073 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:30,073 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:30,073 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:30,073 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,073 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:30,075 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:30,075 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:30,075 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,075 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:30,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:30,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:30,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:30,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:30,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:30,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:30,081 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:30,081 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:30,081 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:30,081 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,081 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:30,083 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:30,084 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,084 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:30,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:30,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:30,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:30,085 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:30,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:30,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,092 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,092 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,092 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:30,092 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:30,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:30,092 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:30,092 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,092 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,092 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,092 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:30,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:30,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:30,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:30,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:30,092 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,093 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:30,094 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:30,194 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:30,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:30,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:30,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:30,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,208 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:30,208 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:30,209 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:30,209 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:30,209 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,209 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,212 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,212 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:30,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:30,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:30,215 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,216 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:30,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:30,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,217 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-10 04:53:30,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,217 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:30,218 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:30,218 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:30,221 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:30,221 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:30,221 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:30,221 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:30,221 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:30,246 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205aa6c5dd28a3d42c6b28f6030ca2ecb48 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205aa6c5dd28a3d42c6b28f6030ca2ecb48 2023-11-10 04:53:30,658 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/99d4a12bec1f45baa4a6687bb0642564 2023-11-10 04:53:30,659 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-11-10 04:53:30,662 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-11-10 04:53:30,672 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/99d4a12bec1f45baa4a6687bb0642564 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/99d4a12bec1f45baa4a6687bb0642564 2023-11-10 04:53:30,689 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/99d4a12bec1f45baa4a6687bb0642564, entries=1, sequenceid=17, filesize=5.0 K 2023-11-10 04:53:30,690 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 469ms, sequenceid=17, compaction requested=true 2023-11-10 04:53:30,691 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:30,691 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:30,691 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:30,692 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:30,692 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,692 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:30,692 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:30,695 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:30,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:30,696 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:30,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:30,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:30,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69047ec1[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:30,704 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:30,704 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,706 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,706 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:30,706 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:30,706 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:30,707 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,707 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:30,708 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,708 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:30,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:30,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:30,709 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:30,709 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:30,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:30,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:30,713 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:30,713 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:30,713 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:30,713 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:30,713 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,713 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:30,718 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,718 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:30,719 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:30,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:30,719 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:30,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:30,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:30,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,722 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,728 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:30,728 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,728 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,728 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,728 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,728 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,729 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,729 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:30,729 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-10 04:53:30,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:30,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:30,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:30,729 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:30,729 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:30,730 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:30,730 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:30,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:30,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:30,830 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:30,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:30,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:30,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:30,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,856 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:30,856 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:30,856 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:30,856 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:30,857 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,857 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,858 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,858 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:30,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:30,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:30,859 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:30,859 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:30,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,860 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:30,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-10 04:53:30,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:30,861 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:30,861 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:30,862 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:30,862 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:30,862 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:30,862 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:30,862 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:30,862 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:31,289 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228f882338fe8df47949db0a30d79c6ed72 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228f882338fe8df47949db0a30d79c6ed72 2023-11-10 04:53:31,351 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:53:31,467 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-10 04:53:31,468 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-11-10 04:53:31,702 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/0bb7d46079ea4f78a680513b6035e60f 2023-11-10 04:53:31,711 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/0bb7d46079ea4f78a680513b6035e60f as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bb7d46079ea4f78a680513b6035e60f 2023-11-10 04:53:31,720 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bb7d46079ea4f78a680513b6035e60f, entries=1, sequenceid=21, filesize=5.0 K 2023-11-10 04:53:31,721 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 859ms, sequenceid=21, compaction requested=true 2023-11-10 04:53:31,721 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:31,721 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:31,722 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:31,722 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:31,722 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,722 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:31,722 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:31,724 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,724 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:31,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:31,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:31,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:31,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:31,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:31,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ca32237[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:31,727 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:31,727 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,728 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,729 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:31,729 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:31,729 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:31,729 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,729 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:31,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:31,731 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:31,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:31,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:31,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:31,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:31,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:31,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:31,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:31,735 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:31,735 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:31,735 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:31,735 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,735 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:31,737 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,737 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:31,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:31,737 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:31,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:31,738 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,739 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:31,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,742 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,743 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,743 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,743 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:31,743 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:31,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:31,743 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,743 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,743 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:31,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-10 04:53:31,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:31,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:31,743 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:31,743 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:31,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:31,744 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:31,744 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:31,844 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:31,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:31,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:31,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:31,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,862 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:31,862 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:31,862 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:31,862 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:31,863 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,863 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,865 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,865 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:31,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:31,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:31,866 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:31,866 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:31,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-10 04:53:31,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:31,867 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:31,867 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:31,868 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:31,868 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:31,868 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:31,868 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:31,868 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:31,868 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:32,294 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512319c35e9f8084b424297e6ee0fd3035a7a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512319c35e9f8084b424297e6ee0fd3035a7a 2023-11-10 04:53:32,707 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/7a4c291f98d24dd381d0c9155bacc932 2023-11-10 04:53:32,718 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/7a4c291f98d24dd381d0c9155bacc932 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/7a4c291f98d24dd381d0c9155bacc932 2023-11-10 04:53:32,728 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/7a4c291f98d24dd381d0c9155bacc932, entries=1, sequenceid=25, filesize=5.0 K 2023-11-10 04:53:32,730 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 862ms, sequenceid=25, compaction requested=true 2023-11-10 04:53:32,730 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:32,730 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:32,730 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:32,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:32,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:32,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:32,734 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,734 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:32,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:32,735 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,735 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:32,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:32,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:32,737 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:32,737 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b59e46a[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:32,737 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:32,737 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,738 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,739 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:32,739 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:32,739 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:32,739 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,739 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:32,741 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:32,741 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,741 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:32,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:32,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:32,741 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:32,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:32,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:32,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:32,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:32,745 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:32,745 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:32,746 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:32,746 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:32,746 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,746 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:32,748 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,748 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:32,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:32,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,748 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:32,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:32,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:32,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:32,749 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:32,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,751 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:32,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:32,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:32,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:32,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:32,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:32,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:32,756 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:32,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:32,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:32,757 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:32,757 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:32,857 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:32,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:32,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:32,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:32,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,869 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:32,869 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:32,870 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:32,870 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:32,870 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,870 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,872 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,872 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:32,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:32,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:32,872 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:32,872 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:32,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,873 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:32,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,874 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,874 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-10 04:53:32,874 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:32,874 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:32,874 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:32,875 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:32,875 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:32,875 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:32,875 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:32,875 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:32,875 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:33,297 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010136c084fce2b442b1a200cf6500706f92 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010136c084fce2b442b1a200cf6500706f92 2023-11-10 04:53:33,706 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/25a50dc178634aafa1c71d0a64b4865c 2023-11-10 04:53:33,715 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/25a50dc178634aafa1c71d0a64b4865c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/25a50dc178634aafa1c71d0a64b4865c 2023-11-10 04:53:33,724 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/25a50dc178634aafa1c71d0a64b4865c, entries=1, sequenceid=29, filesize=5.0 K 2023-11-10 04:53:33,725 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 850ms, sequenceid=29, compaction requested=true 2023-11-10 04:53:33,726 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:33,726 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,727 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:33,727 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:33,732 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,732 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:33,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:33,732 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,732 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:33,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:33,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:33,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:33,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:33,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41d788bc[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:33,734 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:33,735 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,737 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,737 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:33,737 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:33,738 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:33,738 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,738 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:33,739 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:33,739 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,739 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:33,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:33,739 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:33,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:33,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:33,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:33,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:33,743 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:33,743 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:33,743 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:33,743 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,743 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:33,745 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,745 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,746 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:33,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:33,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:33,746 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:33,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:33,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:33,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:33,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,748 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:33,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,751 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,751 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,752 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,752 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:33,752 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,752 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,752 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:33,752 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:33,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:33,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:33,752 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:33,752 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:33,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-10 04:53:33,753 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:33,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:33,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:33,753 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:33,853 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:33,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:33,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:33,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:33,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,865 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:33,865 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:33,866 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:33,866 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:33,866 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,866 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,869 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:33,869 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:33,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:33,869 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:33,869 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:33,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:33,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,870 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,870 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-10 04:53:33,871 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:33,871 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:33,871 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:33,871 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:33,871 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:33,871 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:33,871 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:33,872 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:33,872 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:34,326 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010387e37b9168d04823bc17c51769f00ef3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010387e37b9168d04823bc17c51769f00ef3 2023-11-10 04:53:34,336 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/9d91ef052934467a9c05d03881319718 2023-11-10 04:53:34,344 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/9d91ef052934467a9c05d03881319718 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/9d91ef052934467a9c05d03881319718 2023-11-10 04:53:34,354 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/9d91ef052934467a9c05d03881319718, entries=1, sequenceid=33, filesize=5.0 K 2023-11-10 04:53:34,355 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 483ms, sequenceid=33, compaction requested=true 2023-11-10 04:53:34,355 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:34,355 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:34,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:34,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:34,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:34,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:34,359 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,359 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:34,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:34,359 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,360 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:34,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:34,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:34,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:34,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:34,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@bef0431[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:34,362 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:34,362 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,364 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,364 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:34,364 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:34,364 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:34,364 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,364 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:34,367 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:34,367 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,367 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:34,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:34,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:34,367 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:34,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:34,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:34,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:34,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:34,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:34,372 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:34,372 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:34,372 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:34,372 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,372 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:34,375 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,375 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:34,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,376 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:34,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:34,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:34,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:34,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:34,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,378 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:34,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:34,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,381 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,384 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,385 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,385 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:34,385 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:34,385 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:34,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:34,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:34,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:34,385 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,385 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:34,386 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:34,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:34,386 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,386 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,386 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:34,386 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:34,486 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:34,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:34,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:34,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:34,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,497 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:34,498 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:34,498 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:34,498 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:34,499 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,499 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,500 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,500 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:34,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:34,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:34,501 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:34,501 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:34,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,502 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,502 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-10 04:53:34,502 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:34,503 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:34,503 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:34,504 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:34,504 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:34,504 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:34,504 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:34,504 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:34,504 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:34,924 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090759f8eacee72348cf8023736332d076aa to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015090759f8eacee72348cf8023736332d076aa 2023-11-10 04:53:35,333 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/36027330f54b4449920ed928148fe926 2023-11-10 04:53:35,343 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/36027330f54b4449920ed928148fe926 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/36027330f54b4449920ed928148fe926 2023-11-10 04:53:35,351 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/36027330f54b4449920ed928148fe926, entries=1, sequenceid=37, filesize=5.0 K 2023-11-10 04:53:35,352 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 848ms, sequenceid=37, compaction requested=true 2023-11-10 04:53:35,352 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:35,352 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:35,352 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:35,353 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:35,353 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,353 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:35,353 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:35,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,356 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:35,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:35,357 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,357 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:35,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:35,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:35,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ec83de1[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:35,360 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:35,360 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,364 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,364 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:35,364 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:35,364 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:35,365 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,365 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:35,366 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:35,366 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,366 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:35,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:35,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:35,366 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:35,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:35,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:35,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:35,372 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:35,372 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:35,372 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:35,372 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,372 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:35,381 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,381 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:35,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:35,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,381 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:35,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:35,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:35,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:35,383 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:35,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,387 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:35,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,395 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,395 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,395 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:35,395 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:35,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:35,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:35,395 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:35,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:35,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:35,395 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,396 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,396 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,396 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,396 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-10 04:53:35,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:35,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:35,396 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:35,396 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:35,496 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:35,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:35,506 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:35,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:35,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,509 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:35,509 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:35,509 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-10 04:53:35,509 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-10 04:53:35,510 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,510 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,512 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:35,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:35,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:35,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,513 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,513 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:35,513 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:35,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:35,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,521 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-10 04:53:35,521 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-10 04:53:35,521 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-10 04:53:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:35,522 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:35,522 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:35,522 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. started... 2023-11-10 04:53:35,523 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0cb33d1bbec7f17d50fcde1f31e585e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:35,944 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511205d77fc14060d4962ba595b437511239d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511205d77fc14060d4962ba595b437511239d 2023-11-10 04:53:36,353 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/854d4313161a4060b4523c4c53f841d9 2023-11-10 04:53:36,361 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/854d4313161a4060b4523c4c53f841d9 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/854d4313161a4060b4523c4c53f841d9 2023-11-10 04:53:36,368 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/854d4313161a4060b4523c4c53f841d9, entries=1, sequenceid=41, filesize=5.0 K 2023-11-10 04:53:36,369 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d0cb33d1bbec7f17d50fcde1f31e585e in 846ms, sequenceid=41, compaction requested=true 2023-11-10 04:53:36,369 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:36,369 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:36,369 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:36,369 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:36,370 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,370 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-10 04:53:36,370 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-10 04:53:36,371 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,371 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:36,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:36,372 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,372 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:36,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:36,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:36,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:36,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-10 04:53:36,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a46f0e6[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:36,374 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-10 04:53:36,374 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,375 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,375 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-10 04:53:36,375 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-10 04:53:36,375 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:36,375 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,375 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:36,378 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-10 04:53:36,378 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,378 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:36,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,378 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-10 04:53:36,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:36,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:36,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:36,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:36,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:36,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:36,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-10 04:53:36,382 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-10 04:53:36,382 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:36,382 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:36,382 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,382 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:36,384 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,384 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:36,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:36,384 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:36,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,385 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:36,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:36,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:36,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:36,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,389 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:36,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-10 04:53:36,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-10 04:53:36,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:36,392 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:36,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,392 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:36,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:36,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:36,393 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:36,493 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:36,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:36,500 DEBUG [Listener at localhost/35765] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-10 04:53:36,503 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56642, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-10 04:53:36,507 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:36,524 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-10 04:53:36,524 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-10 04:53:36,527 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:53:36,527 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592616527 2023-11-10 04:53:36,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:53:36,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-11-10 04:53:36,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:36,534 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x1bc1c5ad to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:53:36,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f14c5f8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:53:36,539 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,027 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010359610694320a4b889bcb93dcdb7de1ba to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010359610694320a4b889bcb93dcdb7de1ba 2023-11-10 04:53:37,028 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:37,050 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:53:37,051 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35984, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:53:37,065 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:53:37,068 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56648, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:53:37,116 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/584ffffe391b4830b5c3f3fb197af0a3 first=Optional[r3] last=Optional[r6] 2023-11-10 04:53:37,135 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/584ffffe391b4830b5c3f3fb197af0a3}] 2023-11-10 04:53:37,155 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/584ffffe391b4830b5c3f3fb197af0a3 for inclusion in d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:37,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-10 04:53:37,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:53:37,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:37,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/584ffffe391b4830b5c3f3fb197af0a3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__83m8a6dkh7k8hrd4j617vn73krefqlal73ua2ovt03q4sigmgb1vb88fs9e8pafp/family1/584ffffe391b4830b5c3f3fb197af0a3 2023-11-10 04:53:37,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__83m8a6dkh7k8hrd4j617vn73krefqlal73ua2ovt03q4sigmgb1vb88fs9e8pafp/family1/584ffffe391b4830b5c3f3fb197af0a3 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/75dcbcad6727439fb7bf5834bb6f53ce_SeqId_44_ 2023-11-10 04:53:37,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__83m8a6dkh7k8hrd4j617vn73krefqlal73ua2ovt03q4sigmgb1vb88fs9e8pafp/family1/584ffffe391b4830b5c3f3fb197af0a3 into d0cb33d1bbec7f17d50fcde1f31e585e/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/75dcbcad6727439fb7bf5834bb6f53ce_SeqId_44_ - updating store file list. 2023-11-10 04:53:37,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/75dcbcad6727439fb7bf5834bb6f53ce_SeqId_44_ into d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:37,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__83m8a6dkh7k8hrd4j617vn73krefqlal73ua2ovt03q4sigmgb1vb88fs9e8pafp/family1/584ffffe391b4830b5c3f3fb197af0a3 into d0cb33d1bbec7f17d50fcde1f31e585e/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/75dcbcad6727439fb7bf5834bb6f53ce_SeqId_44_) 2023-11-10 04:53:37,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__83m8a6dkh7k8hrd4j617vn73krefqlal73ua2ovt03q4sigmgb1vb88fs9e8pafp/family1/584ffffe391b4830b5c3f3fb197af0a3 2023-11-10 04:53:37,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-10 04:53:37,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-10 04:53:37,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): d0cb33d1bbec7f17d50fcde1f31e585e/family1 is initiating minor compaction 2023-11-10 04:53:37,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@35b93964; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:53:37,192 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d0cb33d1bbec7f17d50fcde1f31e585e/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:37,193 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/83bc815d2cea41ac833e7e03344aa3af, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bcbd2889e1a480c82257f5241c33448, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/99d4a12bec1f45baa4a6687bb0642564, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bb7d46079ea4f78a680513b6035e60f, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/7a4c291f98d24dd381d0c9155bacc932, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/25a50dc178634aafa1c71d0a64b4865c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/9d91ef052934467a9c05d03881319718, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/36027330f54b4449920ed928148fe926, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/854d4313161a4060b4523c4c53f841d9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/75dcbcad6727439fb7bf5834bb6f53ce_SeqId_44_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp, totalSize=50.2 K 2023-11-10 04:53:37,193 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 83bc815d2cea41ac833e7e03344aa3af, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-10 04:53:37,194 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 0bcbd2889e1a480c82257f5241c33448, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-10 04:53:37,194 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 99d4a12bec1f45baa4a6687bb0642564, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-10 04:53:37,194 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 0bb7d46079ea4f78a680513b6035e60f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-10 04:53:37,194 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 7a4c291f98d24dd381d0c9155bacc932, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-10 04:53:37,194 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 25a50dc178634aafa1c71d0a64b4865c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-10 04:53:37,194 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 9d91ef052934467a9c05d03881319718, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-10 04:53:37,194 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 36027330f54b4449920ed928148fe926, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-10 04:53:37,194 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 854d4313161a4060b4523c4c53f841d9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-10 04:53:37,194 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 75dcbcad6727439fb7bf5834bb6f53ce_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-10 04:53:37,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:37,209 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:37,215 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228f882338fe8df47949db0a30d79c6ed72 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228f882338fe8df47949db0a30d79c6ed72 2023-11-10 04:53:37,216 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512319c35e9f8084b424297e6ee0fd3035a7a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512319c35e9f8084b424297e6ee0fd3035a7a 2023-11-10 04:53:37,218 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010136c084fce2b442b1a200cf6500706f92 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010136c084fce2b442b1a200cf6500706f92 2023-11-10 04:53:37,220 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010387e37b9168d04823bc17c51769f00ef3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010387e37b9168d04823bc17c51769f00ef3 2023-11-10 04:53:37,220 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-11-10 04:53:37,220 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-10 04:53:37,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,246 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): d0cb33d1bbec7f17d50fcde1f31e585e#family1#compaction#10 average throughput is 0.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:53:37,706 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051945a7dd354a4222ad347d9dd685a1b7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512051945a7dd354a4222ad347d9dd685a1b7 2023-11-10 04:53:37,707 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:37,763 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ff2820f633df47e482c258976d365491 first=Optional[r00] last=Optional[r2] 2023-11-10 04:53:37,764 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ff2820f633df47e482c258976d365491}] 2023-11-10 04:53:37,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ff2820f633df47e482c258976d365491 for inclusion in d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:37,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-10 04:53:37,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:53:37,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:37,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ff2820f633df47e482c258976d365491 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3tc2rggrrrjtf5s4io3fqid5r630qgi4uptcknvia24b7ij1e54i48pfhioata9f/family1/ff2820f633df47e482c258976d365491 2023-11-10 04:53:37,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3tc2rggrrrjtf5s4io3fqid5r630qgi4uptcknvia24b7ij1e54i48pfhioata9f/family1/ff2820f633df47e482c258976d365491 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_ 2023-11-10 04:53:37,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3tc2rggrrrjtf5s4io3fqid5r630qgi4uptcknvia24b7ij1e54i48pfhioata9f/family1/ff2820f633df47e482c258976d365491 into d0cb33d1bbec7f17d50fcde1f31e585e/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_ - updating store file list. 2023-11-10 04:53:37,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_ into d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:37,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3tc2rggrrrjtf5s4io3fqid5r630qgi4uptcknvia24b7ij1e54i48pfhioata9f/family1/ff2820f633df47e482c258976d365491 into d0cb33d1bbec7f17d50fcde1f31e585e/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_) 2023-11-10 04:53:37,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3tc2rggrrrjtf5s4io3fqid5r630qgi4uptcknvia24b7ij1e54i48pfhioata9f/family1/ff2820f633df47e482c258976d365491 2023-11-10 04:53:37,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:37,789 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:37,791 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a1c3eeca78954369bbcf502557a298e5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a1c3eeca78954369bbcf502557a298e5 2023-11-10 04:53:37,792 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201fdd9ba7d50e8451fb9b94634a04e8692 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201fdd9ba7d50e8451fb9b94634a04e8692 2023-11-10 04:53:37,794 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205aa6c5dd28a3d42c6b28f6030ca2ecb48 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205aa6c5dd28a3d42c6b28f6030ca2ecb48 2023-11-10 04:53:37,794 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-11-10 04:53:37,794 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-10 04:53:37,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:37,795 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:37,796 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:53:37,797 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-10 04:53:37,799 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.2830 sec 2023-11-10 04:53:37,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-10 04:53:37,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:38,072 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/5a938e4a5b0b4295922e8d883d376da0 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/5a938e4a5b0b4295922e8d883d376da0 2023-11-10 04:53:38,087 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d0cb33d1bbec7f17d50fcde1f31e585e/family1 of d0cb33d1bbec7f17d50fcde1f31e585e into 5a938e4a5b0b4295922e8d883d376da0(size=6.6 K), total size for store is 358.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:53:38,087 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:38,088 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1699592017190; duration=0sec 2023-11-10 04:53:38,089 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:53:38,104 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/83bc815d2cea41ac833e7e03344aa3af, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bcbd2889e1a480c82257f5241c33448, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/99d4a12bec1f45baa4a6687bb0642564, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bb7d46079ea4f78a680513b6035e60f, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/7a4c291f98d24dd381d0c9155bacc932, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/25a50dc178634aafa1c71d0a64b4865c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/9d91ef052934467a9c05d03881319718, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/36027330f54b4449920ed928148fe926, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/854d4313161a4060b4523c4c53f841d9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/75dcbcad6727439fb7bf5834bb6f53ce_SeqId_44_] to archive 2023-11-10 04:53:38,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:38,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/83bc815d2cea41ac833e7e03344aa3af to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/83bc815d2cea41ac833e7e03344aa3af 2023-11-10 04:53:38,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bcbd2889e1a480c82257f5241c33448 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bcbd2889e1a480c82257f5241c33448 2023-11-10 04:53:38,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/99d4a12bec1f45baa4a6687bb0642564 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/99d4a12bec1f45baa4a6687bb0642564 2023-11-10 04:53:38,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bb7d46079ea4f78a680513b6035e60f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/0bb7d46079ea4f78a680513b6035e60f 2023-11-10 04:53:38,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/7a4c291f98d24dd381d0c9155bacc932 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/7a4c291f98d24dd381d0c9155bacc932 2023-11-10 04:53:38,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/25a50dc178634aafa1c71d0a64b4865c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/25a50dc178634aafa1c71d0a64b4865c 2023-11-10 04:53:38,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/9d91ef052934467a9c05d03881319718 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/9d91ef052934467a9c05d03881319718 2023-11-10 04:53:38,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/36027330f54b4449920ed928148fe926 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/36027330f54b4449920ed928148fe926 2023-11-10 04:53:38,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/854d4313161a4060b4523c4c53f841d9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/854d4313161a4060b4523c4c53f841d9 2023-11-10 04:53:38,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/75dcbcad6727439fb7bf5834bb6f53ce_SeqId_44_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/75dcbcad6727439fb7bf5834bb6f53ce_SeqId_44_ 2023-11-10 04:53:38,393 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-11-10 04:53:38,395 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-11-10 04:53:38,395 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-11-10 04:53:38,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-11-10 04:53:38,414 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=d0cb33d1bbec7f17d50fcde1f31e585e, REOPEN/MOVE}] 2023-11-10 04:53:38,417 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=d0cb33d1bbec7f17d50fcde1f31e585e, REOPEN/MOVE 2023-11-10 04:53:38,418 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=d0cb33d1bbec7f17d50fcde1f31e585e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:38,418 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1699592018418"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592018418"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592018418"}]},"ts":"1699592018418"} 2023-11-10 04:53:38,421 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure d0cb33d1bbec7f17d50fcde1f31e585e, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:53:38,580 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:38,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:38,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d0cb33d1bbec7f17d50fcde1f31e585e, disabling compactions & flushes 2023-11-10 04:53:38,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:38,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:38,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. after waiting 0 ms 2023-11-10 04:53:38,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:38,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-11-10 04:53:38,607 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:38,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:38,607 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: d0cb33d1bbec7f17d50fcde1f31e585e to self. 2023-11-10 04:53:38,610 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=d0cb33d1bbec7f17d50fcde1f31e585e, regionState=CLOSED 2023-11-10 04:53:38,610 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1699592018610"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592018610"}]},"ts":"1699592018610"} 2023-11-10 04:53:38,616 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=16, resume processing ppid=15 2023-11-10 04:53:38,616 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure d0cb33d1bbec7f17d50fcde1f31e585e, server=jenkins-hbase4.apache.org,36419,1699592002485 in 192 msec 2023-11-10 04:53:38,617 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:38,617 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=d0cb33d1bbec7f17d50fcde1f31e585e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=true 2023-11-10 04:53:38,767 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=d0cb33d1bbec7f17d50fcde1f31e585e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:38,768 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1699592018767"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592018767"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592018767"}]},"ts":"1699592018767"} 2023-11-10 04:53:38,770 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure d0cb33d1bbec7f17d50fcde1f31e585e, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:53:38,922 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:38,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:38,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d0cb33d1bbec7f17d50fcde1f31e585e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:53:38,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:38,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:38,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:38,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:38,928 INFO [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:38,930 DEBUG [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:38,930 DEBUG [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:38,930 INFO [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0cb33d1bbec7f17d50fcde1f31e585e columnFamilyName family1 2023-11-10 04:53:38,938 DEBUG [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1076a9f70e4840f3a862e8c3cbea0982 2023-11-10 04:53:38,944 DEBUG [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/5a938e4a5b0b4295922e8d883d376da0 2023-11-10 04:53:38,949 DEBUG [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_ 2023-11-10 04:53:38,949 INFO [StoreOpener-d0cb33d1bbec7f17d50fcde1f31e585e-1] regionserver.HStore(310): Store=d0cb33d1bbec7f17d50fcde1f31e585e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:38,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:38,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:38,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:53:38,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d0cb33d1bbec7f17d50fcde1f31e585e; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61489022, jitterRate=-0.0837421715259552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:53:38,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:38,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e., pid=17, masterSystemTime=1699592018922 2023-11-10 04:53:38,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:53:38,958 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-10 04:53:38,959 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-10 04:53:38,960 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1912): d0cb33d1bbec7f17d50fcde1f31e585e/family1 is initiating minor compaction (all files) 2023-11-10 04:53:38,960 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d0cb33d1bbec7f17d50fcde1f31e585e/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:38,960 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1076a9f70e4840f3a862e8c3cbea0982, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/5a938e4a5b0b4295922e8d883d376da0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp, totalSize=358.6 K 2023-11-10 04:53:38,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:38,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:53:38,960 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 1076a9f70e4840f3a862e8c3cbea0982, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878827283 2023-11-10 04:53:38,961 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=d0cb33d1bbec7f17d50fcde1f31e585e, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:38,961 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 5a938e4a5b0b4295922e8d883d376da0, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621227283 2023-11-10 04:53:38,961 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1699592018961"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592018961"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592018961"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592018961"}]},"ts":"1699592018961"} 2023-11-10 04:53:38,961 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878827283 2023-11-10 04:53:38,965 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=17, resume processing ppid=15 2023-11-10 04:53:38,966 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure d0cb33d1bbec7f17d50fcde1f31e585e, server=jenkins-hbase4.apache.org,36419,1699592002485 in 193 msec 2023-11-10 04:53:38,968 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=15, resume processing ppid=14 2023-11-10 04:53:38,968 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=d0cb33d1bbec7f17d50fcde1f31e585e, REOPEN/MOVE in 552 msec 2023-11-10 04:53:38,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=14, resume processing ppid=13 2023-11-10 04:53:38,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 570 msec 2023-11-10 04:53:38,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0020 sec 2023-11-10 04:53:38,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=13 2023-11-10 04:53:38,999 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): d0cb33d1bbec7f17d50fcde1f31e585e#family1#compaction#11 average throughput is 6.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:53:39,004 INFO [Listener at localhost/35765] compactions.TestMobCompactor(278): alter status finished 2023-11-10 04:53:39,006 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-10 04:53:39,009 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-10 04:53:39,013 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-10 04:53:39,015 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592619015 2023-11-10 04:53:39,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:53:39,019 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:53:39,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-11-10 04:53:39,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:39,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x00e894d1 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:53:39,045 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2dd43781, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:53:39,046 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:39,046 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:39,046 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:39,046 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:39,046 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:39,103 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f671f390d62748338d3fc78a58327385 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128f671f390d62748338d3fc78a58327385 2023-11-10 04:53:39,105 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:39,121 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:53:39,123 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35988, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:53:39,133 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:53:39,136 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56654, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:53:39,151 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/cc37bb7cdcc34ef4a0bb159cfca280fb first=Optional[r05] last=Optional[r8] 2023-11-10 04:53:39,152 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=50 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/cc37bb7cdcc34ef4a0bb159cfca280fb}] 2023-11-10 04:53:39,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/cc37bb7cdcc34ef4a0bb159cfca280fb for inclusion in d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:39,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-10 04:53:39,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:53:39,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:39,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/cc37bb7cdcc34ef4a0bb159cfca280fb to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__osm0dq7mi2cjl6rluom2lac6da337sb21ee8o3f1h2o75hatml1ouu7hp4afsc0p/family1/cc37bb7cdcc34ef4a0bb159cfca280fb 2023-11-10 04:53:39,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__osm0dq7mi2cjl6rluom2lac6da337sb21ee8o3f1h2o75hatml1ouu7hp4afsc0p/family1/cc37bb7cdcc34ef4a0bb159cfca280fb as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1ca6096bbbb5438eb4c3e3df0dec1c80_SeqId_52_ 2023-11-10 04:53:39,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__osm0dq7mi2cjl6rluom2lac6da337sb21ee8o3f1h2o75hatml1ouu7hp4afsc0p/family1/cc37bb7cdcc34ef4a0bb159cfca280fb into d0cb33d1bbec7f17d50fcde1f31e585e/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1ca6096bbbb5438eb4c3e3df0dec1c80_SeqId_52_ - updating store file list. 2023-11-10 04:53:39,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1ca6096bbbb5438eb4c3e3df0dec1c80_SeqId_52_ into d0cb33d1bbec7f17d50fcde1f31e585e/family1 2023-11-10 04:53:39,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__osm0dq7mi2cjl6rluom2lac6da337sb21ee8o3f1h2o75hatml1ouu7hp4afsc0p/family1/cc37bb7cdcc34ef4a0bb159cfca280fb into d0cb33d1bbec7f17d50fcde1f31e585e/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1ca6096bbbb5438eb4c3e3df0dec1c80_SeqId_52_) 2023-11-10 04:53:39,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__osm0dq7mi2cjl6rluom2lac6da337sb21ee8o3f1h2o75hatml1ouu7hp4afsc0p/family1/cc37bb7cdcc34ef4a0bb159cfca280fb 2023-11-10 04:53:39,175 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:39,176 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:39,178 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511205d77fc14060d4962ba595b437511239d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511205d77fc14060d4962ba595b437511239d 2023-11-10 04:53:39,180 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ded5070166b1476e8aa4d915b128c68a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ded5070166b1476e8aa4d915b128c68a 2023-11-10 04:53:39,180 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-11-10 04:53:39,180 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-10 04:53:39,180 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:39,180 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:39,180 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-10 04:53:39,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-10 04:53:39,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:53:39,181 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-10 04:53:39,183 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 173 msec 2023-11-10 04:53:39,304 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=257 (was 214) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38586 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51169@0x00e894d1-SendThread(127.0.0.1:51169) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51169@0x1bc1c5ad-SendThread(127.0.0.1:51169) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38554 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38614 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46cb10a8-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_945167450_17 at /127.0.0.1:38788 [Receiving block BP-1289017901-172.31.14.131-1699591997884:blk_1073741865_1041] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46cb10a8-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1289017901-172.31.14.131-1699591997884:blk_1073741865_1041, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x413a950f-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51169@0x1bc1c5ad-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38500 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46cb10a8-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x46cb10a8-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fdf9c6f67a1247a0a877c4e26f27e6ae_del java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38664 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51169@0x00e894d1-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38772 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51169@0x00e894d1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/906541723.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x413a950f-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_945167450_17 at /127.0.0.1:38558 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38626 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46cb10a8-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ResponseProcessor for block BP-1289017901-172.31.14.131-1699591997884:blk_1073741865_1041 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/40996e7a29444296aa2825e0fb078f76 block BP-1289017901-172.31.14.131-1699591997884:blk_1073741865_1041 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:57170 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38564 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x413a950f-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:36419-shortCompactions-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:940) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:880) org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:863) org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:818) org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.finishClose(HFileWriterImpl.java:820) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.close(HFileWriterImpl.java:654) org.apache.hadoop.hbase.regionserver.StoreFileWriter.close(StoreFileWriter.java:359) org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.abortWriter(DefaultCompactor.java:97) org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.performCompaction(DefaultMobStoreCompactor.java:339) org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:337) org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.compact(DefaultMobStoreCompactor.java:108) org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122) org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1473) org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2267) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:625) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:673) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51169@0x1bc1c5ad sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/906541723.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38756 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38528 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Thread-300 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38598 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38538 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=623 (was 505) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=309 (was 330), ProcessCount=170 (was 170), AvailableMemoryMB=11177 (was 11485) 2023-11-10 04:53:39,318 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=257, OpenFileDescriptor=623, MaxFileDescriptor=60000, SystemLoadAverage=309, ProcessCount=170, AvailableMemoryMB=11175 2023-11-10 04:53:39,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-10 04:53:39,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:39,324 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:53:39,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-11-10 04:53:39,326 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:53:39,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-10 04:53:39,330 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:53:39,331 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c empty. 2023-11-10 04:53:39,331 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:53:39,332 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-11-10 04:53:39,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-10 04:53:39,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-10 04:53:39,741 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-10 04:53:39,743 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3d66950151f7b8ec803595548ad6d16c, NAME => 'testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:53:39,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-10 04:53:40,151 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:40,151 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 3d66950151f7b8ec803595548ad6d16c, disabling compactions & flushes 2023-11-10 04:53:40,152 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:40,152 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:40,152 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. after waiting 0 ms 2023-11-10 04:53:40,152 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:40,152 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:40,152 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:40,154 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:53:40,156 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1699592020155"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592020155"}]},"ts":"1699592020155"} 2023-11-10 04:53:40,157 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-10 04:53:40,159 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:53:40,159 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592020159"}]},"ts":"1699592020159"} 2023-11-10 04:53:40,160 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-11-10 04:53:40,164 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=3d66950151f7b8ec803595548ad6d16c, ASSIGN}] 2023-11-10 04:53:40,166 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=3d66950151f7b8ec803595548ad6d16c, ASSIGN 2023-11-10 04:53:40,166 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=3d66950151f7b8ec803595548ad6d16c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:53:40,246 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/.tmp/family1/40996e7a29444296aa2825e0fb078f76 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/40996e7a29444296aa2825e0fb078f76 2023-11-10 04:53:40,253 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in d0cb33d1bbec7f17d50fcde1f31e585e/family1 of d0cb33d1bbec7f17d50fcde1f31e585e into 40996e7a29444296aa2825e0fb078f76(size=177.2 K), total size for store is 182.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:53:40,253 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:53:40,253 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e., storeName=d0cb33d1bbec7f17d50fcde1f31e585e/family1, priority=13, startTime=1699592018957; duration=1sec 2023-11-10 04:53:40,253 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:53:40,302 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1076a9f70e4840f3a862e8c3cbea0982, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/5a938e4a5b0b4295922e8d883d376da0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_] to archive 2023-11-10 04:53:40,304 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:40,306 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1076a9f70e4840f3a862e8c3cbea0982 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/1076a9f70e4840f3a862e8c3cbea0982 2023-11-10 04:53:40,308 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/5a938e4a5b0b4295922e8d883d376da0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/5a938e4a5b0b4295922e8d883d376da0 2023-11-10 04:53:40,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/family1/6d8a9178ead34c0aaaf14cdb35bcdb9e_SeqId_46_ 2023-11-10 04:53:40,317 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=3d66950151f7b8ec803595548ad6d16c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:40,317 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1699592020317"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592020317"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592020317"}]},"ts":"1699592020317"} 2023-11-10 04:53:40,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 3d66950151f7b8ec803595548ad6d16c, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:53:40,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-10 04:53:40,472 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:40,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:40,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3d66950151f7b8ec803595548ad6d16c, NAME => 'testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:53:40,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:53:40,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:40,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:53:40,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:53:40,478 INFO [StoreOpener-3d66950151f7b8ec803595548ad6d16c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:53:40,480 DEBUG [StoreOpener-3d66950151f7b8ec803595548ad6d16c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1 2023-11-10 04:53:40,480 DEBUG [StoreOpener-3d66950151f7b8ec803595548ad6d16c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1 2023-11-10 04:53:40,481 INFO [StoreOpener-3d66950151f7b8ec803595548ad6d16c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d66950151f7b8ec803595548ad6d16c columnFamilyName family1 2023-11-10 04:53:40,483 INFO [StoreOpener-3d66950151f7b8ec803595548ad6d16c-1] regionserver.HStore(310): Store=3d66950151f7b8ec803595548ad6d16c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:40,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:53:40,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:53:40,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:53:40,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:53:40,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3d66950151f7b8ec803595548ad6d16c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74570311, jitterRate=0.11118422448635101}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:53:40,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:40,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c., pid=21, masterSystemTime=1699592020472 2023-11-10 04:53:40,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:40,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:40,495 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=3d66950151f7b8ec803595548ad6d16c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:40,495 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1699592020495"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592020495"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592020495"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592020495"}]},"ts":"1699592020495"} 2023-11-10 04:53:40,499 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=21, resume processing ppid=20 2023-11-10 04:53:40,499 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 3d66950151f7b8ec803595548ad6d16c, server=jenkins-hbase4.apache.org,36419,1699592002485 in 178 msec 2023-11-10 04:53:40,501 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=20, resume processing ppid=19 2023-11-10 04:53:40,501 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=3d66950151f7b8ec803595548ad6d16c, ASSIGN in 335 msec 2023-11-10 04:53:40,502 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:53:40,502 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592020502"}]},"ts":"1699592020502"} 2023-11-10 04:53:40,503 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-11-10 04:53:40,507 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:53:40,508 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1860 sec 2023-11-10 04:53:41,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-10 04:53:41,433 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-11-10 04:53:41,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:41,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:41,469 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:41,469 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:41,469 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:41,469 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:41,470 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:41,470 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:41,472 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:41,472 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:41,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:41,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:41,472 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:41,472 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:41,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:41,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:41,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:41,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:41,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:41,474 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:41,474 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:41,474 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:41,475 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:41,478 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:41,478 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:41,478 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:41,478 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:41,479 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-10 04:53:41,901 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b59a99b2b11a488181250a940b3de48b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130b59a99b2b11a488181250a940b3de48b 2023-11-10 04:53:42,306 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/98b6e9859e9e4cd39328767929365612 2023-11-10 04:53:42,315 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/98b6e9859e9e4cd39328767929365612 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/98b6e9859e9e4cd39328767929365612 2023-11-10 04:53:42,326 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/98b6e9859e9e4cd39328767929365612, entries=1000, sequenceid=5, filesize=166.0 K 2023-11-10 04:53:42,329 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 850ms, sequenceid=5, compaction requested=false 2023-11-10 04:53:42,329 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:42,330 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:42,330 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:42,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:42,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:42,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:42,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:42,332 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,332 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:42,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:42,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:42,334 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,334 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:42,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:42,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:42,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:42,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1729177a[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:42,336 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:42,336 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,337 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,338 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,338 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:42,339 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:42,339 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:42,339 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:42,342 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:42,342 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:42,342 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:42,343 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:42,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:42,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:42,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:42,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:42,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:42,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:42,349 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:42,349 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:42,349 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:42,349 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,349 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:42,353 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,353 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:42,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:42,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,353 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:42,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:42,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:42,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:42,354 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:42,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:42,357 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,366 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,366 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:42,366 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:42,366 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,366 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:42,367 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:42,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:42,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:42,367 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:42,367 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,367 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,367 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,368 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:42,368 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:42,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:42,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:42,468 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:42,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:42,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:42,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:42,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,477 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:42,477 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:42,478 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:42,478 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:42,478 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,479 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,480 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,480 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:42,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:42,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:42,480 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:42,480 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:42,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:42,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,481 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,482 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:53:42,482 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:42,482 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:42,482 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:42,482 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:42,482 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:42,482 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:42,483 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:42,483 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:42,483 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-10 04:53:42,898 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112891ba724e1c0f4adb8e3e6cefc45c43d8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112891ba724e1c0f4adb8e3e6cefc45c43d8 2023-11-10 04:53:43,304 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/f2d0166f0e2a44b1910e062d66419b40 2023-11-10 04:53:43,311 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/f2d0166f0e2a44b1910e062d66419b40 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f2d0166f0e2a44b1910e062d66419b40 2023-11-10 04:53:43,317 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f2d0166f0e2a44b1910e062d66419b40, entries=1, sequenceid=9, filesize=5.0 K 2023-11-10 04:53:43,319 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 836ms, sequenceid=9, compaction requested=false 2023-11-10 04:53:43,319 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:43,319 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:43,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:43,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:43,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:43,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,321 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:43,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:43,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:43,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:43,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64145722[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:43,323 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:43,323 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,325 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:43,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:43,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:43,325 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,325 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:43,327 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,327 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:43,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:43,328 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:43,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:43,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:43,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:43,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:43,331 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:43,331 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:43,331 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:43,331 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:43,331 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,331 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:43,333 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,333 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,334 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:43,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:43,334 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:43,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,336 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:43,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,338 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,338 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,338 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:43,338 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,339 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:43,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:43,339 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:43,339 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:43,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:43,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:43,339 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,339 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,339 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:43,340 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:43,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:43,340 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:43,440 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:43,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:43,448 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:43,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:43,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,454 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:43,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:43,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:43,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:43,456 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,456 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,458 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,458 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:43,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:43,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:43,459 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:43,459 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:43,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,459 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:43,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,460 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,460 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:53:43,460 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:43,461 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:43,461 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:43,461 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:43,461 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:43,461 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:43,461 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:43,462 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:43,462 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:43,877 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120132c6d6965cd14fb981e392a98a192e9a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120132c6d6965cd14fb981e392a98a192e9a 2023-11-10 04:53:44,133 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:53:44,283 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/a7f5e3a49f124ea9bd68f4f783d80395 2023-11-10 04:53:44,289 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/a7f5e3a49f124ea9bd68f4f783d80395 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/a7f5e3a49f124ea9bd68f4f783d80395 2023-11-10 04:53:44,295 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/a7f5e3a49f124ea9bd68f4f783d80395, entries=1, sequenceid=13, filesize=4.9 K 2023-11-10 04:53:44,296 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 834ms, sequenceid=13, compaction requested=true 2023-11-10 04:53:44,296 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:44,296 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:44,296 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:44,296 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:44,296 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,296 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:44,296 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:44,299 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,299 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:44,299 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:44,299 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:44,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:44,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:44,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:44,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b6c5974[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:44,301 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:44,301 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,303 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,303 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:44,303 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:44,303 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:44,303 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,303 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:44,305 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:44,305 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:44,305 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,305 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:44,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:44,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:44,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:44,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:44,308 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:44,308 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:44,308 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:44,308 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,308 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:44,310 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,310 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:44,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:44,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,310 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:44,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:44,311 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,312 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:44,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:44,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:44,317 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:44,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:44,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:44,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:44,317 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:44,317 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,318 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,318 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,318 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:44,318 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:44,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:44,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:44,418 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:44,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:44,425 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:44,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:44,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,426 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:44,426 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:44,427 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:44,427 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:44,427 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,428 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,429 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:44,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:44,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:44,429 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:44,429 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:44,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:44,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,431 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:53:44,431 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:44,431 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:44,431 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:44,431 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:44,431 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:44,431 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:44,431 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:44,431 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:44,432 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:44,846 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054c8ac652334745729a177fe6992d04e8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512054c8ac652334745729a177fe6992d04e8 2023-11-10 04:53:45,251 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/5811977f98a04ce39d25067ff5075c7e 2023-11-10 04:53:45,258 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/5811977f98a04ce39d25067ff5075c7e as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5811977f98a04ce39d25067ff5075c7e 2023-11-10 04:53:45,263 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5811977f98a04ce39d25067ff5075c7e, entries=1, sequenceid=17, filesize=4.9 K 2023-11-10 04:53:45,264 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 832ms, sequenceid=17, compaction requested=true 2023-11-10 04:53:45,264 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:45,264 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:45,264 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:45,264 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:45,264 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,264 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:45,264 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:45,266 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,266 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:45,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:45,267 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,267 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:45,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:45,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:45,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:45,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:45,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66704f3f[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:45,268 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:45,268 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,270 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,270 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:45,270 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:45,270 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:45,270 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,270 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:45,276 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:45,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:45,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:45,276 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:45,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:45,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:45,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:45,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:45,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:45,279 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:45,279 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:45,279 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:45,279 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,279 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:45,281 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,281 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:45,281 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:45,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:45,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:45,282 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,284 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:45,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,286 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,287 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,287 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,287 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:45,287 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:45,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:45,287 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:45,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:45,287 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:45,287 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:45,287 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,287 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:45,288 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:45,288 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:45,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:45,388 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:45,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:45,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:45,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:45,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,396 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:45,396 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:45,397 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:45,397 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:45,397 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,397 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,399 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:45,399 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:45,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:45,399 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:45,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,399 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:45,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:45,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,400 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:53:45,400 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:45,400 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:45,400 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:45,401 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:45,401 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:45,401 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:45,401 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:45,401 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:45,401 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:45,816 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228ea004b4ac27147fe82b7057aeae90531 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228ea004b4ac27147fe82b7057aeae90531 2023-11-10 04:53:46,223 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/5dce3979af3542619767739e4e566fc1 2023-11-10 04:53:46,230 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/5dce3979af3542619767739e4e566fc1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5dce3979af3542619767739e4e566fc1 2023-11-10 04:53:46,236 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5dce3979af3542619767739e4e566fc1, entries=1, sequenceid=21, filesize=4.9 K 2023-11-10 04:53:46,237 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 836ms, sequenceid=21, compaction requested=true 2023-11-10 04:53:46,237 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:46,237 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:46,237 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:46,237 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:46,237 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,238 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:46,238 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:46,241 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,241 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:46,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:46,241 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:46,241 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:46,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:46,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:46,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:46,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39bd164c[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:46,243 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:46,243 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:46,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:46,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:46,246 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,246 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:46,247 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,247 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:46,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:46,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:46,247 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:46,247 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:46,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:46,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:46,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:46,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:46,251 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:46,251 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:46,251 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:46,251 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:46,251 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,251 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:46,253 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,253 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:46,253 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:46,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:46,254 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,255 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:46,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,259 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,259 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:46,259 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:46,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:46,259 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:46,259 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:46,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:46,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:46,259 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,260 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,260 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,260 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:46,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:46,260 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,260 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:46,260 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:46,361 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:46,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:46,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:46,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:46,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,369 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:46,369 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:46,369 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:46,369 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:46,370 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,370 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,371 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,371 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:46,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:46,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:46,372 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:46,372 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:46,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,372 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:46,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,373 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,373 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:53:46,373 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:46,373 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:46,373 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:46,373 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:46,374 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:46,374 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:46,374 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:46,374 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:46,374 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:46,792 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c92a29ba05154f1ebe82e43403fd92ce to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c92a29ba05154f1ebe82e43403fd92ce 2023-11-10 04:53:47,198 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/cafce3b1843249b4bb6c8fe3a63dc560 2023-11-10 04:53:47,204 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/cafce3b1843249b4bb6c8fe3a63dc560 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/cafce3b1843249b4bb6c8fe3a63dc560 2023-11-10 04:53:47,208 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/cafce3b1843249b4bb6c8fe3a63dc560, entries=1, sequenceid=25, filesize=4.9 K 2023-11-10 04:53:47,209 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 835ms, sequenceid=25, compaction requested=true 2023-11-10 04:53:47,210 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:47,210 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:47,210 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:47,210 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:47,210 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,210 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:47,210 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:47,211 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,212 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:47,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:47,212 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,212 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:47,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:47,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:47,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:47,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:47,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f56fb1e[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:47,213 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:47,214 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,215 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,215 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:47,215 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:47,215 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:47,215 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,215 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:47,216 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,216 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:47,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:47,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:47,216 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:47,216 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:47,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:47,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:47,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:47,219 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:47,219 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:47,219 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:47,220 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,220 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:47,222 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,222 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:47,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:47,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,222 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:47,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:47,223 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:47,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:47,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:47,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,224 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:47,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,227 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,227 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:47,227 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:47,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:47,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-10 04:53:47,228 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,228 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:47,228 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:47,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:47,228 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:47,228 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,228 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,228 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:47,229 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:47,229 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:47,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:47,329 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:47,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:47,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:47,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:47,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,338 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:47,338 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:47,338 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:47,338 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:47,339 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,339 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,340 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:47,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:47,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:47,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,341 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:53:47,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:47,343 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:47,343 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:47,343 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:47,758 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e1fa20896abe4e88b832dc032f25f313 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101e1fa20896abe4e88b832dc032f25f313 2023-11-10 04:53:48,164 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/2ab8c947eb78464382d95fb28d3b7457 2023-11-10 04:53:48,170 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/2ab8c947eb78464382d95fb28d3b7457 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/2ab8c947eb78464382d95fb28d3b7457 2023-11-10 04:53:48,175 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/2ab8c947eb78464382d95fb28d3b7457, entries=1, sequenceid=29, filesize=4.9 K 2023-11-10 04:53:48,176 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 833ms, sequenceid=29, compaction requested=true 2023-11-10 04:53:48,176 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:48,176 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:48,181 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,181 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:48,181 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,181 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:48,183 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:48,183 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2628aa79[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:48,183 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:48,183 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,184 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,184 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:48,184 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:48,184 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:48,184 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,184 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:48,187 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,187 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:48,187 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:48,187 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:48,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:48,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:48,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:48,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:48,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:48,190 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:48,190 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:48,190 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:48,190 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,190 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:48,192 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,192 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,192 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:48,193 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,194 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,197 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,197 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,197 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:48,197 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:48,197 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:48,197 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:48,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:48,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:48,197 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,198 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,198 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,198 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:48,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:48,198 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:48,198 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:48,298 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:48,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:48,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:48,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:48,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,309 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:48,309 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:48,309 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:48,309 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:48,310 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,310 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,312 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,312 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:48,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:48,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:48,312 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,312 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:48,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,313 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:48,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,313 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,314 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:53:48,314 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,314 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:48,314 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:48,314 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:48,314 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:48,314 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:48,314 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:48,315 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:48,315 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:48,332 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010333cd5cdebca44488a5edde81fc89ddf2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010333cd5cdebca44488a5edde81fc89ddf2 2023-11-10 04:53:48,738 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/76dc9d9a5c114dee95ba64b9154b4c0b 2023-11-10 04:53:48,744 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/76dc9d9a5c114dee95ba64b9154b4c0b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/76dc9d9a5c114dee95ba64b9154b4c0b 2023-11-10 04:53:48,749 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/76dc9d9a5c114dee95ba64b9154b4c0b, entries=1, sequenceid=33, filesize=4.9 K 2023-11-10 04:53:48,750 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 435ms, sequenceid=33, compaction requested=true 2023-11-10 04:53:48,750 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:48,750 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:48,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:48,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:48,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:48,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:48,752 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,752 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:48,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:48,752 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,752 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:48,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:48,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42c50034[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:48,754 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:48,754 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,756 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:48,756 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,756 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:48,757 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,757 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:48,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:48,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:48,757 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:48,758 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:48,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:48,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:48,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:48,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:48,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:48,761 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:48,761 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:48,761 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:48,761 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,761 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:48,763 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,763 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,763 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:48,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:48,764 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:48,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:48,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,767 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,770 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:48,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:48,770 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:48,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:48,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-10 04:53:48,770 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,770 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:48,770 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:48,771 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,771 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:48,772 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:48,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:48,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:48,772 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,772 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,772 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,772 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,872 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:48,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:48,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:48,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:48,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,881 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:48,881 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:48,881 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:48,881 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:48,882 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,882 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,883 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,883 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:48,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:48,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:48,883 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:48,883 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:48,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:48,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,884 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,884 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:53:48,885 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:48,885 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:48,885 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:48,885 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:48,885 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:48,885 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:48,885 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:48,885 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:48,885 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:49,303 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509074bf904456ce346bd9fdfc8ccadf7271c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509074bf904456ce346bd9fdfc8ccadf7271c 2023-11-10 04:53:49,310 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/eac9dbcbf1df4b5dac89f6b1fd00100b 2023-11-10 04:53:49,316 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/eac9dbcbf1df4b5dac89f6b1fd00100b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/eac9dbcbf1df4b5dac89f6b1fd00100b 2023-11-10 04:53:49,323 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/eac9dbcbf1df4b5dac89f6b1fd00100b, entries=1, sequenceid=37, filesize=4.9 K 2023-11-10 04:53:49,324 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 439ms, sequenceid=37, compaction requested=true 2023-11-10 04:53:49,324 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:49,324 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:49,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:49,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:49,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:49,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:49,327 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:49,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:49,327 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,327 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,327 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:49,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:49,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:49,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:49,330 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:49,330 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@131b7156[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:49,330 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:49,330 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,332 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,332 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:49,332 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:49,333 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:49,333 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,333 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:49,335 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:49,335 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:49,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,335 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:49,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:49,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:49,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:49,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:49,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:49,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:49,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:49,339 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:49,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:49,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:49,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,340 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:49,342 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:49,342 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,342 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:49,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:49,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:49,343 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:49,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,344 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:49,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,355 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:49,355 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:49,355 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,355 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:49,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:49,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:49,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:49,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-10 04:53:49,355 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:49,356 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,356 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,356 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,356 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:49,356 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,357 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:49,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:49,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:49,357 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,457 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:49,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:49,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:49,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:49,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,465 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:49,465 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:49,465 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:53:49,466 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:49,466 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,466 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,467 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:49,467 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:49,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:49,468 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,468 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:49,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:49,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,469 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,469 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:53:49,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,469 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:53:49,469 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:49,471 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:53:49,471 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:49,471 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:49,471 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:49,471 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. started... 2023-11-10 04:53:49,471 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d66950151f7b8ec803595548ad6d16c 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:49,490 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e0d7f42abb9845c1887ad790368e27fa to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120e0d7f42abb9845c1887ad790368e27fa 2023-11-10 04:53:49,498 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/f009449121b349cabfafd6e79fa6e147 2023-11-10 04:53:49,504 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/f009449121b349cabfafd6e79fa6e147 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f009449121b349cabfafd6e79fa6e147 2023-11-10 04:53:49,511 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f009449121b349cabfafd6e79fa6e147, entries=1, sequenceid=41, filesize=4.9 K 2023-11-10 04:53:49,512 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 3d66950151f7b8ec803595548ad6d16c in 41ms, sequenceid=41, compaction requested=true 2023-11-10 04:53:49,512 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:49,512 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:49,512 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:49,512 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:49,512 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,512 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:53:49,512 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:53:49,514 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,514 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:49,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:49,515 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,515 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:49,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:49,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:49,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:49,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:53:49,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b5de620[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:49,517 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:49,517 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,520 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,520 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:49,520 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:53:49,520 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:49,520 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,520 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:49,521 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,521 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:53:49,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:49,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:49,522 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:49,522 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:53:49,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:49,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:49,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:49,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:49,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:49,526 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:53:49,526 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:49,526 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:49,526 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,526 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:49,528 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,528 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:49,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:49,528 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:49,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:49,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:49,529 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:49,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:49,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,531 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:49,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,535 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,535 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,535 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,535 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:49,535 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:49,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:49,535 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:49,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:49,535 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,535 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-10 04:53:49,536 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:49,536 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,536 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,535 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:49,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:49,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:49,536 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:49,538 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:49,638 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:49,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:49,641 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-11-10 04:53:49,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:53:49,643 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:53:49,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:53:49,644 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592629644 2023-11-10 04:53:49,645 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:53:49,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-11-10 04:53:49,646 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:53:49,648 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x5f030c5d to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:53:49,652 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@53e74003, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:53:49,652 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:53:50,084 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b81cceb446674df1a1735f4617f9b109 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103b81cceb446674df1a1735f4617f9b109 2023-11-10 04:53:50,086 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:53:50,100 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:53:50,101 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36656, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:53:50,110 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:53:50,111 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39256, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:53:50,125 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/0e630cba7464477e89ceaa3239abd2ee first=Optional[r5] last=Optional[r6] 2023-11-10 04:53:50,126 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/0e630cba7464477e89ceaa3239abd2ee}] 2023-11-10 04:53:50,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/0e630cba7464477e89ceaa3239abd2ee for inclusion in 3d66950151f7b8ec803595548ad6d16c/family1 2023-11-10 04:53:50,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-11-10 04:53:50,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:53:50,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:50,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/0e630cba7464477e89ceaa3239abd2ee to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2mtfgarta5b2muaolqkb2frou8k7lvq1866u809eomjshlluj5i9egb13iuokuk7/family1/0e630cba7464477e89ceaa3239abd2ee 2023-11-10 04:53:50,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2mtfgarta5b2muaolqkb2frou8k7lvq1866u809eomjshlluj5i9egb13iuokuk7/family1/0e630cba7464477e89ceaa3239abd2ee as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/d4d2da51ad4a4ee8b5143ab72ef64f6d_SeqId_44_ 2023-11-10 04:53:50,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2mtfgarta5b2muaolqkb2frou8k7lvq1866u809eomjshlluj5i9egb13iuokuk7/family1/0e630cba7464477e89ceaa3239abd2ee into 3d66950151f7b8ec803595548ad6d16c/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/d4d2da51ad4a4ee8b5143ab72ef64f6d_SeqId_44_ - updating store file list. 2023-11-10 04:53:50,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/d4d2da51ad4a4ee8b5143ab72ef64f6d_SeqId_44_ into 3d66950151f7b8ec803595548ad6d16c/family1 2023-11-10 04:53:50,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2mtfgarta5b2muaolqkb2frou8k7lvq1866u809eomjshlluj5i9egb13iuokuk7/family1/0e630cba7464477e89ceaa3239abd2ee into 3d66950151f7b8ec803595548ad6d16c/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/d4d2da51ad4a4ee8b5143ab72ef64f6d_SeqId_44_) 2023-11-10 04:53:50,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2mtfgarta5b2muaolqkb2frou8k7lvq1866u809eomjshlluj5i9egb13iuokuk7/family1/0e630cba7464477e89ceaa3239abd2ee 2023-11-10 04:53:50,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-10 04:53:50,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-10 04:53:50,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 3d66950151f7b8ec803595548ad6d16c/family1 is initiating minor compaction 2023-11-10 04:53:50,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@742c6157; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:53:50,153 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3d66950151f7b8ec803595548ad6d16c/family1 in testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:50,153 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f2d0166f0e2a44b1910e062d66419b40, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/a7f5e3a49f124ea9bd68f4f783d80395, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5811977f98a04ce39d25067ff5075c7e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5dce3979af3542619767739e4e566fc1, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/cafce3b1843249b4bb6c8fe3a63dc560, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/2ab8c947eb78464382d95fb28d3b7457, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/76dc9d9a5c114dee95ba64b9154b4c0b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/eac9dbcbf1df4b5dac89f6b1fd00100b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f009449121b349cabfafd6e79fa6e147, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/d4d2da51ad4a4ee8b5143ab72ef64f6d_SeqId_44_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp, totalSize=49.7 K 2023-11-10 04:53:50,153 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting f2d0166f0e2a44b1910e062d66419b40, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-10 04:53:50,153 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting a7f5e3a49f124ea9bd68f4f783d80395, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-10 04:53:50,153 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 5811977f98a04ce39d25067ff5075c7e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-10 04:53:50,153 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 5dce3979af3542619767739e4e566fc1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-10 04:53:50,154 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting cafce3b1843249b4bb6c8fe3a63dc560, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-10 04:53:50,154 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 2ab8c947eb78464382d95fb28d3b7457, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-10 04:53:50,154 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 76dc9d9a5c114dee95ba64b9154b4c0b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-10 04:53:50,154 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting eac9dbcbf1df4b5dac89f6b1fd00100b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-10 04:53:50,154 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting f009449121b349cabfafd6e79fa6e147, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-10 04:53:50,154 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting d4d2da51ad4a4ee8b5143ab72ef64f6d_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-10 04:53:50,155 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:53:50,157 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:50,160 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101e1fa20896abe4e88b832dc032f25f313 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101e1fa20896abe4e88b832dc032f25f313 2023-11-10 04:53:50,161 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010333cd5cdebca44488a5edde81fc89ddf2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010333cd5cdebca44488a5edde81fc89ddf2 2023-11-10 04:53:50,161 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-11-10 04:53:50,161 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-11-10 04:53:50,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:53:50,162 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:53:50,196 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3d66950151f7b8ec803595548ad6d16c#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:53:51,017 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/6ee10c753666446e8f74ad977fbd3806 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/6ee10c753666446e8f74ad977fbd3806 2023-11-10 04:53:51,017 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311452b188b7074f33b6b42bcc4b6b225f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512311452b188b7074f33b6b42bcc4b6b225f 2023-11-10 04:53:51,019 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:53:51,029 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 3d66950151f7b8ec803595548ad6d16c/family1 of 3d66950151f7b8ec803595548ad6d16c into 6ee10c753666446e8f74ad977fbd3806(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:53:51,029 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:51,030 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1699592030153; duration=0sec 2023-11-10 04:53:51,030 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:53:51,051 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/442b456141fe4c0d88865c5c8c3af3d5 first=Optional[r1] last=Optional[r4] 2023-11-10 04:53:51,052 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/442b456141fe4c0d88865c5c8c3af3d5}] 2023-11-10 04:53:51,054 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/442b456141fe4c0d88865c5c8c3af3d5 for inclusion in 3d66950151f7b8ec803595548ad6d16c/family1 2023-11-10 04:53:51,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-11-10 04:53:51,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:53:51,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:51,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/442b456141fe4c0d88865c5c8c3af3d5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__tcfe23quhr01g8lq38kfr7kij5lcvbhjic1n4puf4ge9fdhlpo4erkiu2b3l64pd/family1/442b456141fe4c0d88865c5c8c3af3d5 2023-11-10 04:53:51,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__tcfe23quhr01g8lq38kfr7kij5lcvbhjic1n4puf4ge9fdhlpo4erkiu2b3l64pd/family1/442b456141fe4c0d88865c5c8c3af3d5 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/4713415049c14e73943ec45b28de2e9b_SeqId_47_ 2023-11-10 04:53:51,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__tcfe23quhr01g8lq38kfr7kij5lcvbhjic1n4puf4ge9fdhlpo4erkiu2b3l64pd/family1/442b456141fe4c0d88865c5c8c3af3d5 into 3d66950151f7b8ec803595548ad6d16c/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/4713415049c14e73943ec45b28de2e9b_SeqId_47_ - updating store file list. 2023-11-10 04:53:51,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/4713415049c14e73943ec45b28de2e9b_SeqId_47_ into 3d66950151f7b8ec803595548ad6d16c/family1 2023-11-10 04:53:51,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__tcfe23quhr01g8lq38kfr7kij5lcvbhjic1n4puf4ge9fdhlpo4erkiu2b3l64pd/family1/442b456141fe4c0d88865c5c8c3af3d5 into 3d66950151f7b8ec803595548ad6d16c/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/4713415049c14e73943ec45b28de2e9b_SeqId_47_) 2023-11-10 04:53:51,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__tcfe23quhr01g8lq38kfr7kij5lcvbhjic1n4puf4ge9fdhlpo4erkiu2b3l64pd/family1/442b456141fe4c0d88865c5c8c3af3d5 2023-11-10 04:53:51,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-10 04:53:51,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-10 04:53:51,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 3d66950151f7b8ec803595548ad6d16c/family1 is initiating minor compaction (all files) 2023-11-10 04:53:51,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@c0f34d3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:53:51,071 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3d66950151f7b8ec803595548ad6d16c/family1 in testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:53:51,071 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/98b6e9859e9e4cd39328767929365612, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/6ee10c753666446e8f74ad977fbd3806, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/4713415049c14e73943ec45b28de2e9b_SeqId_47_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp, totalSize=178.1 K 2023-11-10 04:53:51,072 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 98b6e9859e9e4cd39328767929365612, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878827283 2023-11-10 04:53:51,072 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 6ee10c753666446e8f74ad977fbd3806, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621227283 2023-11-10 04:53:51,073 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 4713415049c14e73943ec45b28de2e9b_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965227283 2023-11-10 04:53:51,077 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:53:51,081 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:51,085 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120132c6d6965cd14fb981e392a98a192e9a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120132c6d6965cd14fb981e392a98a192e9a 2023-11-10 04:53:51,088 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512054c8ac652334745729a177fe6992d04e8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512054c8ac652334745729a177fe6992d04e8 2023-11-10 04:53:51,090 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228ea004b4ac27147fe82b7057aeae90531 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228ea004b4ac27147fe82b7057aeae90531 2023-11-10 04:53:51,092 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c92a29ba05154f1ebe82e43403fd92ce to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c92a29ba05154f1ebe82e43403fd92ce 2023-11-10 04:53:51,092 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-11-10 04:53:51,092 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-11-10 04:53:51,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:53:51,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:53:51,105 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3d66950151f7b8ec803595548ad6d16c#family1#compaction#23 average throughput is 8.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:53:51,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f2d0166f0e2a44b1910e062d66419b40, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/a7f5e3a49f124ea9bd68f4f783d80395, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5811977f98a04ce39d25067ff5075c7e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5dce3979af3542619767739e4e566fc1, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/cafce3b1843249b4bb6c8fe3a63dc560, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/2ab8c947eb78464382d95fb28d3b7457, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/76dc9d9a5c114dee95ba64b9154b4c0b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/eac9dbcbf1df4b5dac89f6b1fd00100b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f009449121b349cabfafd6e79fa6e147, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/d4d2da51ad4a4ee8b5143ab72ef64f6d_SeqId_44_] to archive 2023-11-10 04:53:51,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:51,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f2d0166f0e2a44b1910e062d66419b40 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f2d0166f0e2a44b1910e062d66419b40 2023-11-10 04:53:51,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/a7f5e3a49f124ea9bd68f4f783d80395 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/a7f5e3a49f124ea9bd68f4f783d80395 2023-11-10 04:53:51,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5811977f98a04ce39d25067ff5075c7e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5811977f98a04ce39d25067ff5075c7e 2023-11-10 04:53:51,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5dce3979af3542619767739e4e566fc1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5dce3979af3542619767739e4e566fc1 2023-11-10 04:53:51,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/cafce3b1843249b4bb6c8fe3a63dc560 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/cafce3b1843249b4bb6c8fe3a63dc560 2023-11-10 04:53:51,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/2ab8c947eb78464382d95fb28d3b7457 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/2ab8c947eb78464382d95fb28d3b7457 2023-11-10 04:53:51,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/76dc9d9a5c114dee95ba64b9154b4c0b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/76dc9d9a5c114dee95ba64b9154b4c0b 2023-11-10 04:53:51,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/eac9dbcbf1df4b5dac89f6b1fd00100b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/eac9dbcbf1df4b5dac89f6b1fd00100b 2023-11-10 04:53:51,126 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f009449121b349cabfafd6e79fa6e147 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/f009449121b349cabfafd6e79fa6e147 2023-11-10 04:53:51,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/d4d2da51ad4a4ee8b5143ab72ef64f6d_SeqId_44_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/d4d2da51ad4a4ee8b5143ab72ef64f6d_SeqId_44_ 2023-11-10 04:53:51,608 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/.tmp/family1/a5aad7b6d3c1412aa080f688a9a4f76f as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/a5aad7b6d3c1412aa080f688a9a4f76f 2023-11-10 04:53:51,615 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 3d66950151f7b8ec803595548ad6d16c/family1 of 3d66950151f7b8ec803595548ad6d16c into a5aad7b6d3c1412aa080f688a9a4f76f(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:53:51,615 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:51,615 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1699592031071; duration=0sec 2023-11-10 04:53:51,615 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:53:51,698 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/98b6e9859e9e4cd39328767929365612, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/6ee10c753666446e8f74ad977fbd3806, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/4713415049c14e73943ec45b28de2e9b_SeqId_47_] to archive 2023-11-10 04:53:51,698 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:51,700 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/98b6e9859e9e4cd39328767929365612 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/98b6e9859e9e4cd39328767929365612 2023-11-10 04:53:51,702 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/6ee10c753666446e8f74ad977fbd3806 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/6ee10c753666446e8f74ad977fbd3806 2023-11-10 04:53:51,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/4713415049c14e73943ec45b28de2e9b_SeqId_47_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/4713415049c14e73943ec45b28de2e9b_SeqId_47_ 2023-11-10 04:53:52,024 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113001ec5df8381244bb8f5cedcbe1d12228 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113001ec5df8381244bb8f5cedcbe1d12228 2023-11-10 04:53:52,025 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:53:52,050 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/56cc3230ea8a4cecbbd8520a7183c6a4 first=Optional[r00] last=Optional[r8] 2023-11-10 04:53:52,051 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/56cc3230ea8a4cecbbd8520a7183c6a4}] 2023-11-10 04:53:52,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/56cc3230ea8a4cecbbd8520a7183c6a4 for inclusion in 3d66950151f7b8ec803595548ad6d16c/family1 2023-11-10 04:53:52,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-11-10 04:53:52,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:53:52,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:53:52,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/56cc3230ea8a4cecbbd8520a7183c6a4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__33u0la8tuvnhm3dg2uiji28rfq1q30mtdeaccr3gk3m3kve1b5e6f7s5m3ggk9s1/family1/56cc3230ea8a4cecbbd8520a7183c6a4 2023-11-10 04:53:52,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__33u0la8tuvnhm3dg2uiji28rfq1q30mtdeaccr3gk3m3kve1b5e6f7s5m3ggk9s1/family1/56cc3230ea8a4cecbbd8520a7183c6a4 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5260e1244a444b2da1165d02f5017cf1_SeqId_50_ 2023-11-10 04:53:52,060 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__33u0la8tuvnhm3dg2uiji28rfq1q30mtdeaccr3gk3m3kve1b5e6f7s5m3ggk9s1/family1/56cc3230ea8a4cecbbd8520a7183c6a4 into 3d66950151f7b8ec803595548ad6d16c/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5260e1244a444b2da1165d02f5017cf1_SeqId_50_ - updating store file list. 2023-11-10 04:53:52,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5260e1244a444b2da1165d02f5017cf1_SeqId_50_ into 3d66950151f7b8ec803595548ad6d16c/family1 2023-11-10 04:53:52,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__33u0la8tuvnhm3dg2uiji28rfq1q30mtdeaccr3gk3m3kve1b5e6f7s5m3ggk9s1/family1/56cc3230ea8a4cecbbd8520a7183c6a4 into 3d66950151f7b8ec803595548ad6d16c/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/family1/5260e1244a444b2da1165d02f5017cf1_SeqId_50_) 2023-11-10 04:53:52,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithMonthlyPolicy__33u0la8tuvnhm3dg2uiji28rfq1q30mtdeaccr3gk3m3kve1b5e6f7s5m3ggk9s1/family1/56cc3230ea8a4cecbbd8520a7183c6a4 2023-11-10 04:53:52,067 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:53:52,068 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:53:52,069 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120e0d7f42abb9845c1887ad790368e27fa to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120e0d7f42abb9845c1887ad790368e27fa 2023-11-10 04:53:52,070 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112891ba724e1c0f4adb8e3e6cefc45c43d8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112891ba724e1c0f4adb8e3e6cefc45c43d8 2023-11-10 04:53:52,071 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130b59a99b2b11a488181250a940b3de48b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130b59a99b2b11a488181250a940b3de48b 2023-11-10 04:53:52,071 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-11-10 04:53:52,071 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-10 04:53:52,072 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:53:52,072 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:53:52,072 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:53:52,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:53:52,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:53:52,073 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:53:52,075 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.4320 sec 2023-11-10 04:53:52,230 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=259 (was 257) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38586 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46e0359d-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:41912 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/cluster_48cecc52-0985-53ae-674b-7482adaea2d3/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:42076 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51169@0x5f030c5d-SendThread(127.0.0.1:51169) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51169@0x5f030c5d sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/906541723.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:42070 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46e0359d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46e0359d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_945167450_17 at /127.0.0.1:42054 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/cluster_48cecc52-0985-53ae-674b-7482adaea2d3/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46e0359d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:42036 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_945167450_17 at /127.0.0.1:41958 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46e0359d-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:42044 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46e0359d-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x537b9561-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:41930 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:41994 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x46e0359d-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:42204 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:41982 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:42146 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:38756 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:41996 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:51169@0x5f030c5d-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:42080 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272709818_17 at /127.0.0.1:42116 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=645 (was 623) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=278 (was 309), ProcessCount=170 (was 170), AvailableMemoryMB=11104 (was 11175) 2023-11-10 04:53:52,246 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=259, OpenFileDescriptor=645, MaxFileDescriptor=60000, SystemLoadAverage=278, ProcessCount=170, AvailableMemoryMB=11103 2023-11-10 04:53:52,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-10 04:53:52,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:52,252 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:53:52,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-11-10 04:53:52,253 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:53:52,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-10 04:53:52,255 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:53:52,255 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2 empty. 2023-11-10 04:53:52,256 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:53:52,256 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-11-10 04:53:52,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-10 04:53:52,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-10 04:53:52,665 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-10 04:53:52,667 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 422f46c0e9462ab44880fe4aeb8fb7a2, NAME => 'testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:53:52,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-10 04:53:53,073 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:53,074 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 422f46c0e9462ab44880fe4aeb8fb7a2, disabling compactions & flushes 2023-11-10 04:53:53,074 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:53,074 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:53,074 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. after waiting 0 ms 2023-11-10 04:53:53,074 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:53,074 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:53,074 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:53:53,076 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:53:53,077 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1699592033077"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592033077"}]},"ts":"1699592033077"} 2023-11-10 04:53:53,079 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-10 04:53:53,080 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:53:53,080 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592033080"}]},"ts":"1699592033080"} 2023-11-10 04:53:53,081 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-11-10 04:53:53,085 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=422f46c0e9462ab44880fe4aeb8fb7a2, ASSIGN}] 2023-11-10 04:53:53,087 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=422f46c0e9462ab44880fe4aeb8fb7a2, ASSIGN 2023-11-10 04:53:53,088 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=422f46c0e9462ab44880fe4aeb8fb7a2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:53:53,239 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=422f46c0e9462ab44880fe4aeb8fb7a2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:53,239 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1699592033238"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592033238"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592033238"}]},"ts":"1699592033238"} 2023-11-10 04:53:53,241 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 422f46c0e9462ab44880fe4aeb8fb7a2, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:53:53,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-10 04:53:53,393 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:53,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:53,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 422f46c0e9462ab44880fe4aeb8fb7a2, NAME => 'testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:53:53,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:53:53,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:53:53,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:53:53,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:53:53,398 INFO [StoreOpener-422f46c0e9462ab44880fe4aeb8fb7a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:53:53,399 DEBUG [StoreOpener-422f46c0e9462ab44880fe4aeb8fb7a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1 2023-11-10 04:53:53,399 DEBUG [StoreOpener-422f46c0e9462ab44880fe4aeb8fb7a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1 2023-11-10 04:53:53,400 INFO [StoreOpener-422f46c0e9462ab44880fe4aeb8fb7a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 422f46c0e9462ab44880fe4aeb8fb7a2 columnFamilyName family1 2023-11-10 04:53:53,401 INFO [StoreOpener-422f46c0e9462ab44880fe4aeb8fb7a2-1] regionserver.HStore(310): Store=422f46c0e9462ab44880fe4aeb8fb7a2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:53:53,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:53:53,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:53:53,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:53:53,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:53:53,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 422f46c0e9462ab44880fe4aeb8fb7a2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68738323, jitterRate=0.024280831217765808}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:53:53,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:53:53,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2., pid=25, masterSystemTime=1699592033392 2023-11-10 04:53:53,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:53,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:53,410 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=422f46c0e9462ab44880fe4aeb8fb7a2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:53,410 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1699592033410"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592033410"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592033410"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592033410"}]},"ts":"1699592033410"} 2023-11-10 04:53:53,414 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=25, resume processing ppid=24 2023-11-10 04:53:53,414 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 422f46c0e9462ab44880fe4aeb8fb7a2, server=jenkins-hbase4.apache.org,36419,1699592002485 in 171 msec 2023-11-10 04:53:53,415 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=24, resume processing ppid=23 2023-11-10 04:53:53,416 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=422f46c0e9462ab44880fe4aeb8fb7a2, ASSIGN in 329 msec 2023-11-10 04:53:53,416 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:53:53,416 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592033416"}]},"ts":"1699592033416"} 2023-11-10 04:53:53,418 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-11-10 04:53:53,420 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:53:53,421 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1710 sec 2023-11-10 04:53:54,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-10 04:53:54,359 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-11-10 04:53:54,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:54,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:54,382 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:54,382 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:54,382 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:53:54,382 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:54,382 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:54,383 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:54,384 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:54,384 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:54,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:54,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:54,384 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:54,384 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:54,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:54,385 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:54,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:54,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:54,385 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:54,385 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:54,386 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:53:54,386 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:54,386 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:53:54,386 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:54,386 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:54,386 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:54,386 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:53:54,387 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-10 04:53:54,667 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:54,811 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307f2597d08b3a4ef7969f2a1c3b71d8bf to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511307f2597d08b3a4ef7969f2a1c3b71d8bf 2023-11-10 04:53:55,215 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/a386c16b63b64cb1b4d1d1acb3d4a911 2023-11-10 04:53:55,221 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/a386c16b63b64cb1b4d1d1acb3d4a911 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/a386c16b63b64cb1b4d1d1acb3d4a911 2023-11-10 04:53:55,226 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/a386c16b63b64cb1b4d1d1acb3d4a911, entries=1000, sequenceid=5, filesize=165.1 K 2023-11-10 04:53:55,227 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 839ms, sequenceid=5, compaction requested=false 2023-11-10 04:53:55,227 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:53:55,227 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:55,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:55,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:55,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:53:55,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:53:55,230 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,230 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:55,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:55,230 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,231 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:55,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:55,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:55,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:53:55,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38c5f450[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:55,232 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:55,232 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,233 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,233 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:55,233 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:53:55,233 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:55,233 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,233 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:55,235 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:53:55,235 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,235 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:55,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:55,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:55,235 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:53:55,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:55,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:55,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:55,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:55,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:55,238 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:53:55,238 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:55,238 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:55,238 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,238 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:55,239 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,239 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,239 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:55,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:55,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:55,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:55,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:55,240 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:55,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:55,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,241 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:55,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:55,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:55,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:55,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:55,245 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,245 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:55,246 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:55,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:55,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:55,346 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:55,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:55,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:55,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:55,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,352 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:55,353 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:55,353 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:53:55,353 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:55,353 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,353 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,354 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:55,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:55,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:55,355 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,355 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:55,355 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:55,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:55,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,356 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:53:55,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:55,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:53:55,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:55,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:53:55,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:55,356 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:55,356 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:55,357 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:53:55,357 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-10 04:53:55,770 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128efa70c8b1e714c72ac63af7bc8e3db11 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128efa70c8b1e714c72ac63af7bc8e3db11 2023-11-10 04:53:56,175 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/cd8136a290454513bb4d31935fa6b324 2023-11-10 04:53:56,181 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/cd8136a290454513bb4d31935fa6b324 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/cd8136a290454513bb4d31935fa6b324 2023-11-10 04:53:56,186 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/cd8136a290454513bb4d31935fa6b324, entries=1, sequenceid=9, filesize=5.0 K 2023-11-10 04:53:56,187 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 830ms, sequenceid=9, compaction requested=false 2023-11-10 04:53:56,187 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:53:56,187 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:56,187 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:56,187 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:56,187 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,187 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:53:56,187 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:53:56,193 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,193 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:56,193 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,193 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:56,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:56,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:53:56,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@526b87dc[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:56,195 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:56,195 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,197 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,197 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:56,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:53:56,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:56,198 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,198 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:56,199 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,199 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:53:56,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:56,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:56,199 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:56,199 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:53:56,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:56,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:56,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:56,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:56,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:56,202 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:53:56,202 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:56,202 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:56,202 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,202 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:56,204 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,204 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:56,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:56,204 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:56,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:56,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:56,205 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:56,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:56,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,206 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:56,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,208 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,208 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:56,208 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:56,208 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:56,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-10 04:53:56,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:56,209 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:56,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:56,208 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,208 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:56,209 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,209 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,209 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,209 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,209 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:56,209 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:56,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:56,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:56,309 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:56,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:56,314 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:56,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:56,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,316 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:56,316 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:56,316 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:53:56,316 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:56,317 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,317 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,319 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,319 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:56,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:56,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:56,320 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:56,320 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:56,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:56,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,320 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,321 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:53:56,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:56,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:53:56,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:56,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:53:56,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:56,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:56,321 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:56,321 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:53:56,321 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:56,735 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e1e6ccaf3d9340a79677efde0d63fb88 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201e1e6ccaf3d9340a79677efde0d63fb88 2023-11-10 04:53:57,142 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/d9ed9534448542b28b02f78e8f960921 2023-11-10 04:53:57,147 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/d9ed9534448542b28b02f78e8f960921 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/d9ed9534448542b28b02f78e8f960921 2023-11-10 04:53:57,152 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/d9ed9534448542b28b02f78e8f960921, entries=1, sequenceid=13, filesize=4.9 K 2023-11-10 04:53:57,153 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 832ms, sequenceid=13, compaction requested=true 2023-11-10 04:53:57,153 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:53:57,153 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:57,153 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:57,153 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:57,153 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,153 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:53:57,153 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:53:57,155 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,155 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:57,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:57,155 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,155 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:57,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:57,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:57,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:57,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:53:57,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ab2cd06[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:57,157 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:57,157 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,158 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,159 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:57,159 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:53:57,159 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,159 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:57,159 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:57,160 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,160 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:53:57,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:57,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:57,160 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:57,160 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:53:57,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:57,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:57,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:57,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:57,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:57,163 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:53:57,163 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:57,163 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:57,163 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,163 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:57,166 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,166 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:57,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:57,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,167 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:57,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:57,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:57,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:57,167 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,168 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:57,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,170 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:57,171 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:57,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:57,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:57,171 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:57,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:57,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:57,271 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:57,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:57,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:57,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:57,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,278 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:57,278 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:57,278 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:53:57,278 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:57,279 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,279 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,280 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:57,280 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:57,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:57,281 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:57,281 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:57,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,281 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:57,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,282 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,282 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:53:57,282 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:57,282 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:53:57,282 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:57,282 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:53:57,282 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:57,282 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:57,282 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:57,282 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:53:57,282 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:57,696 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ff91b81ffbee461ea0e6886f4eed1475 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205ff91b81ffbee461ea0e6886f4eed1475 2023-11-10 04:53:58,102 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/57f86f085b6e42c68dd572881e2531fe 2023-11-10 04:53:58,107 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/57f86f085b6e42c68dd572881e2531fe as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/57f86f085b6e42c68dd572881e2531fe 2023-11-10 04:53:58,111 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/57f86f085b6e42c68dd572881e2531fe, entries=1, sequenceid=17, filesize=4.9 K 2023-11-10 04:53:58,112 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 830ms, sequenceid=17, compaction requested=true 2023-11-10 04:53:58,112 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:53:58,112 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:58,112 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:58,112 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:58,112 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,112 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:53:58,112 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:53:58,114 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,114 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:58,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:58,115 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,115 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:58,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:58,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:58,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:58,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:53:58,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e6a5496[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:58,117 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:58,117 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,119 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,119 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:58,119 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:53:58,119 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:58,119 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,119 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:58,121 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,121 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:53:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:58,121 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:58,121 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:53:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:58,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:58,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:58,123 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:58,123 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:53:58,123 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:58,124 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:58,124 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,124 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:58,125 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,125 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,125 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:58,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:58,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:58,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:58,126 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:58,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:58,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:58,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,127 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:58,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:58,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,133 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:58,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:58,133 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-10 04:53:58,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:58,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:58,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:58,133 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:58,134 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:58,134 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:58,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:58,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:58,234 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:58,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:58,239 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:58,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:58,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,241 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:58,241 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:58,241 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:53:58,241 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:58,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,243 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:58,243 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:58,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:58,243 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:58,244 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:58,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:58,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,244 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:53:58,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:58,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:53:58,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:58,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:53:58,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:58,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:58,245 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:58,245 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:53:58,245 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:58,658 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512280f3e366e974147efa658903874747493 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512280f3e366e974147efa658903874747493 2023-11-10 04:53:58,928 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:53:59,064 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/9400b3f4cab240e58d344795e691dd87 2023-11-10 04:53:59,070 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/9400b3f4cab240e58d344795e691dd87 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/9400b3f4cab240e58d344795e691dd87 2023-11-10 04:53:59,075 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/9400b3f4cab240e58d344795e691dd87, entries=1, sequenceid=21, filesize=4.9 K 2023-11-10 04:53:59,075 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 830ms, sequenceid=21, compaction requested=true 2023-11-10 04:53:59,076 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:53:59,076 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:59,076 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:53:59,076 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:53:59,076 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,076 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:53:59,076 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:53:59,078 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,078 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:59,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:59,078 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,078 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:53:59,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:59,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:59,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:59,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:53:59,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@399a427f[Count = 0] remaining members to acquire global barrier 2023-11-10 04:53:59,080 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:53:59,080 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,081 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:53:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:53:59,081 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:53:59,081 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,081 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:53:59,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:53:59,084 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:59,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:59,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:59,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:53:59,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:59,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:59,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:59,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:53:59,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:53:59,086 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:53:59,086 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:53:59,086 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:53:59,086 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,086 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:53:59,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:53:59,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:53:59,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:59,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:53:59,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:59,089 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:59,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:53:59,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,090 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:53:59,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,093 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,093 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:59,093 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:59,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:59,093 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:53:59,093 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,093 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:53:59,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:59,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:53:59,093 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:53:59,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:53:59,093 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,094 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,094 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:53:59,094 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:53:59,094 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,194 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:53:59,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:53:59,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:53:59,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:53:59,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,201 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:53:59,201 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:59,201 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:53:59,201 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:53:59,202 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,202 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,203 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:53:59,203 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:53:59,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:53:59,203 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:53:59,203 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:53:59,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,204 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:53:59,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,204 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:53:59,204 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:53:59,204 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:53:59,205 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:53:59,205 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:53:59,205 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:53:59,205 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:53:59,205 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:53:59,205 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:53:59,205 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:53:59,619 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317254794c14104a11bc56dde3d92a1953 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512317254794c14104a11bc56dde3d92a1953 2023-11-10 04:54:00,025 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/c01b79f89a48422ea1a1b01de601d41c 2023-11-10 04:54:00,031 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/c01b79f89a48422ea1a1b01de601d41c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/c01b79f89a48422ea1a1b01de601d41c 2023-11-10 04:54:00,036 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/c01b79f89a48422ea1a1b01de601d41c, entries=1, sequenceid=25, filesize=4.9 K 2023-11-10 04:54:00,037 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 832ms, sequenceid=25, compaction requested=true 2023-11-10 04:54:00,037 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:54:00,037 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:00,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:00,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:00,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:00,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:00,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,040 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:00,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:00,040 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,040 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:00,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:00,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:00,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:00,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:00,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@374167f7[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:00,041 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:00,042 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,043 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,043 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:00,043 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:00,043 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:00,043 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,043 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:00,044 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,044 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:00,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:00,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:00,044 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:00,044 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:00,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:00,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:00,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:00,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:00,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:00,047 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:00,047 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:00,047 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:00,047 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,047 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:00,048 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,048 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:00,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:00,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,048 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:00,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:00,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:00,049 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:00,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:00,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,050 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:00,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,053 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,053 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,053 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:00,053 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:00,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:54:00,054 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:00,053 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:00,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:00,053 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,053 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:00,054 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,054 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,054 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:00,054 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:00,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:00,054 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:00,154 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:00,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:00,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:00,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:00,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,161 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:54:00,161 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:00,161 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:00,161 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:00,162 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,162 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,163 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:00,163 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:00,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:00,163 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,163 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:00,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:00,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,164 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:00,164 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,164 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:00,164 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:00,164 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:00,165 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:00,165 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:00,165 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:00,165 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:54:00,165 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:00,579 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101626a639a25014f64ac4f79dca48671d6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101626a639a25014f64ac4f79dca48671d6 2023-11-10 04:54:00,984 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/4682b8d048d74ba0982caa3ad0f41de2 2023-11-10 04:54:00,990 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/4682b8d048d74ba0982caa3ad0f41de2 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/4682b8d048d74ba0982caa3ad0f41de2 2023-11-10 04:54:00,995 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/4682b8d048d74ba0982caa3ad0f41de2, entries=1, sequenceid=29, filesize=4.9 K 2023-11-10 04:54:00,996 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 831ms, sequenceid=29, compaction requested=true 2023-11-10 04:54:00,996 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:54:00,996 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:00,996 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:00,996 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:00,996 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,996 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:00,996 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:00,998 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,998 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:00,999 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:00,999 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:00,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:01,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:01,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:01,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b9746e4[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:01,000 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:01,000 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,002 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,002 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:01,002 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:01,002 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:01,002 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,002 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:01,005 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,005 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:01,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:01,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:01,005 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:01,005 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:01,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:01,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:01,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:01,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:01,007 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:01,007 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:01,008 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:01,008 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:01,008 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,008 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:01,014 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,014 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:01,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:01,014 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:01,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:01,015 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:01,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:01,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:01,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,016 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:01,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,018 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,019 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:01,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:01,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:54:01,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:01,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:01,019 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:01,120 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:01,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:01,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:01,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:01,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,127 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:54:01,127 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:01,127 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:01,127 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:01,127 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,127 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,128 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:01,128 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:01,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:01,129 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,129 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:01,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,129 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:01,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,130 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:01,130 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,130 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:01,130 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:01,130 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:01,130 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:01,130 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:01,130 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:01,130 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:54:01,130 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:01,545 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033507a3bf65f84974aadec08f2864d4a3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601033507a3bf65f84974aadec08f2864d4a3 2023-11-10 04:54:01,951 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/475e38a9c06f4fa484f922f1ec558cf7 2023-11-10 04:54:01,956 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/475e38a9c06f4fa484f922f1ec558cf7 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/475e38a9c06f4fa484f922f1ec558cf7 2023-11-10 04:54:01,961 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/475e38a9c06f4fa484f922f1ec558cf7, entries=1, sequenceid=33, filesize=4.9 K 2023-11-10 04:54:01,962 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 832ms, sequenceid=33, compaction requested=true 2023-11-10 04:54:01,962 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:54:01,962 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:01,962 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:01,962 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:01,962 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,962 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:01,962 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:01,965 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,965 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:01,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:01,965 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,965 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:01,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:01,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:01,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:01,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:01,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47256be4[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:01,967 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:01,967 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,968 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:01,968 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:01,968 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:01,968 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,968 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:01,969 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,969 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:01,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:01,969 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:01,970 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:01,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:01,972 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:01,972 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:01,972 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:01,972 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:01,972 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,972 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:01,974 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,974 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:01,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:01,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,974 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:01,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:01,974 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:01,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:01,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:01,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,976 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:01,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:01,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:54:01,979 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:01,979 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:01,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:01,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:01,979 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:01,980 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,080 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:02,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:02,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:02,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:02,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,087 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:54:02,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:02,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:02,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:02,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,089 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:02,089 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:02,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:02,089 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,089 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:02,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,090 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:02,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,090 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,091 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:02,091 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:02,091 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:02,091 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:02,091 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:54:02,091 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:02,505 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907b689621bc0a6411a9e8a2b221e5e37a9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907b689621bc0a6411a9e8a2b221e5e37a9 2023-11-10 04:54:02,912 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/f7876694099f4c35bdb417530e739328 2023-11-10 04:54:02,918 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/f7876694099f4c35bdb417530e739328 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/f7876694099f4c35bdb417530e739328 2023-11-10 04:54:02,922 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/f7876694099f4c35bdb417530e739328, entries=1, sequenceid=37, filesize=4.9 K 2023-11-10 04:54:02,923 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 832ms, sequenceid=37, compaction requested=true 2023-11-10 04:54:02,923 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:54:02,923 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:02,923 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:02,923 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:02,923 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,923 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:02,923 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:02,925 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,925 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:02,925 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,925 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:02,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:02,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:02,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:02,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cc750c5[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:02,927 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:02,927 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,928 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,928 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:02,928 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:02,929 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:02,929 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,929 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:02,931 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:02,931 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,931 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:02,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:02,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:02,931 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:02,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:02,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:02,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:02,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:02,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:02,933 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:02,934 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:02,934 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:02,934 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,934 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:02,935 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,935 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:02,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:02,935 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:02,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:02,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:02,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:02,936 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:02,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,937 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:02,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,939 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:02,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:54:02,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,940 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:02,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:02,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:02,940 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:02,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:02,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:03,040 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:03,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:03,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:03,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:03,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,047 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:54:03,047 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:03,047 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:03,048 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:03,048 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,048 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,050 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,050 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:03,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:03,050 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,050 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,051 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:03,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:03,051 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:03,052 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:03,052 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. started... 2023-11-10 04:54:03,052 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 422f46c0e9462ab44880fe4aeb8fb7a2 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:03,465 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e6e0b8ac062e49678a15aeb11c07378c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120e6e0b8ac062e49678a15aeb11c07378c 2023-11-10 04:54:03,871 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/84d59798828a4ed5bb290ca014fc7b6d 2023-11-10 04:54:03,876 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/84d59798828a4ed5bb290ca014fc7b6d as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/84d59798828a4ed5bb290ca014fc7b6d 2023-11-10 04:54:03,881 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/84d59798828a4ed5bb290ca014fc7b6d, entries=1, sequenceid=41, filesize=4.9 K 2023-11-10 04:54:03,882 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 422f46c0e9462ab44880fe4aeb8fb7a2 in 830ms, sequenceid=41, compaction requested=true 2023-11-10 04:54:03,882 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:54:03,882 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:03,882 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:03,882 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:03,882 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,882 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:03,882 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:03,884 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,885 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:03,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:03,885 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:03,885 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:03,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:03,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:03,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:03,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1085f475[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:03,886 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:03,886 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,890 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,890 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:03,890 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:03,890 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:03,890 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,890 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:03,892 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,892 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:03,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:03,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:03,892 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:03,892 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:03,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:03,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:03,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:03,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:03,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:03,894 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:03,894 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:03,894 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:03,894 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,894 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:03,896 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,896 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,896 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:03,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:03,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:03,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:03,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:03,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:03,896 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:03,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,898 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:03,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,901 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,901 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:03,901 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,902 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:03,902 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-10 04:54:03,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:03,902 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:03,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:03,902 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:03,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:03,902 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,902 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,902 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:03,902 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:03,902 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:03,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:03,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:04,002 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:04,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:04,004 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-11-10 04:54:04,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:54:04,007 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:54:04,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:04,008 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592644008 2023-11-10 04:54:04,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-10 04:54:04,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-11-10 04:54:04,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:04,013 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x6b04a755 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:04,017 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c8f2b75, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:04,018 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:04,864 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035a98c0875fcc42c08b2dadf1e1a6dd9f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601035a98c0875fcc42c08b2dadf1e1a6dd9f 2023-11-10 04:54:04,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:04,878 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:54:04,879 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33180, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:54:04,886 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:54:04,887 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52588, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:54:04,897 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b3170cfd2c224b23844eca6619168337 first=Optional[r3] last=Optional[r6] 2023-11-10 04:54:04,897 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b3170cfd2c224b23844eca6619168337}] 2023-11-10 04:54:04,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b3170cfd2c224b23844eca6619168337 for inclusion in 422f46c0e9462ab44880fe4aeb8fb7a2/family1 2023-11-10 04:54:04,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-10 04:54:04,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:04,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:54:04,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b3170cfd2c224b23844eca6619168337 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__d3aa4j7aegkrjfm1jm42u0oc9jbpv85l9bhaecndm06i5pnqfoc4v8277uha8bbm/family1/b3170cfd2c224b23844eca6619168337 2023-11-10 04:54:04,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__d3aa4j7aegkrjfm1jm42u0oc9jbpv85l9bhaecndm06i5pnqfoc4v8277uha8bbm/family1/b3170cfd2c224b23844eca6619168337 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/8bc490f73ef84c8f9bf0f7b655eb9873_SeqId_44_ 2023-11-10 04:54:04,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__d3aa4j7aegkrjfm1jm42u0oc9jbpv85l9bhaecndm06i5pnqfoc4v8277uha8bbm/family1/b3170cfd2c224b23844eca6619168337 into 422f46c0e9462ab44880fe4aeb8fb7a2/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/8bc490f73ef84c8f9bf0f7b655eb9873_SeqId_44_ - updating store file list. 2023-11-10 04:54:04,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/8bc490f73ef84c8f9bf0f7b655eb9873_SeqId_44_ into 422f46c0e9462ab44880fe4aeb8fb7a2/family1 2023-11-10 04:54:04,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__d3aa4j7aegkrjfm1jm42u0oc9jbpv85l9bhaecndm06i5pnqfoc4v8277uha8bbm/family1/b3170cfd2c224b23844eca6619168337 into 422f46c0e9462ab44880fe4aeb8fb7a2/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/8bc490f73ef84c8f9bf0f7b655eb9873_SeqId_44_) 2023-11-10 04:54:04,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__d3aa4j7aegkrjfm1jm42u0oc9jbpv85l9bhaecndm06i5pnqfoc4v8277uha8bbm/family1/b3170cfd2c224b23844eca6619168337 2023-11-10 04:54:04,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-10 04:54:04,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-10 04:54:04,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 422f46c0e9462ab44880fe4aeb8fb7a2/family1 is initiating minor compaction 2023-11-10 04:54:04,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@69c39f97; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:04,916 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 422f46c0e9462ab44880fe4aeb8fb7a2/family1 in testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:04,917 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/cd8136a290454513bb4d31935fa6b324, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/d9ed9534448542b28b02f78e8f960921, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/57f86f085b6e42c68dd572881e2531fe, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/9400b3f4cab240e58d344795e691dd87, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/c01b79f89a48422ea1a1b01de601d41c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/4682b8d048d74ba0982caa3ad0f41de2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/475e38a9c06f4fa484f922f1ec558cf7, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/f7876694099f4c35bdb417530e739328, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/84d59798828a4ed5bb290ca014fc7b6d, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/8bc490f73ef84c8f9bf0f7b655eb9873_SeqId_44_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp, totalSize=50.0 K 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting cd8136a290454513bb4d31935fa6b324, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting d9ed9534448542b28b02f78e8f960921, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 57f86f085b6e42c68dd572881e2531fe, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 9400b3f4cab240e58d344795e691dd87, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting c01b79f89a48422ea1a1b01de601d41c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 4682b8d048d74ba0982caa3ad0f41de2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 475e38a9c06f4fa484f922f1ec558cf7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting f7876694099f4c35bdb417530e739328, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 84d59798828a4ed5bb290ca014fc7b6d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-10 04:54:04,917 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 8bc490f73ef84c8f9bf0f7b655eb9873_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-10 04:54:04,918 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:04,920 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:04,922 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512280f3e366e974147efa658903874747493 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512280f3e366e974147efa658903874747493 2023-11-10 04:54:04,923 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512317254794c14104a11bc56dde3d92a1953 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512317254794c14104a11bc56dde3d92a1953 2023-11-10 04:54:04,925 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101626a639a25014f64ac4f79dca48671d6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101626a639a25014f64ac4f79dca48671d6 2023-11-10 04:54:04,926 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601033507a3bf65f84974aadec08f2864d4a3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601033507a3bf65f84974aadec08f2864d4a3 2023-11-10 04:54:04,926 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-11-10 04:54:04,926 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-10 04:54:04,927 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:04,927 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:04,927 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:04,927 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:04,927 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:04,928 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:04,928 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:04,928 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:04,961 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 422f46c0e9462ab44880fe4aeb8fb7a2#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:54:05,378 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205cbebcb01441f44ef82551a88f69757ea to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205cbebcb01441f44ef82551a88f69757ea 2023-11-10 04:54:05,379 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:05,380 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/ec00424ab53d4e5dacde85981c354372 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/ec00424ab53d4e5dacde85981c354372 2023-11-10 04:54:05,386 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 422f46c0e9462ab44880fe4aeb8fb7a2/family1 of 422f46c0e9462ab44880fe4aeb8fb7a2 into ec00424ab53d4e5dacde85981c354372(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:54:05,386 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:54:05,386 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1699592044916; duration=0sec 2023-11-10 04:54:05,386 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:05,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/cd8136a290454513bb4d31935fa6b324, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/d9ed9534448542b28b02f78e8f960921, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/57f86f085b6e42c68dd572881e2531fe, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/9400b3f4cab240e58d344795e691dd87, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/c01b79f89a48422ea1a1b01de601d41c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/4682b8d048d74ba0982caa3ad0f41de2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/475e38a9c06f4fa484f922f1ec558cf7, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/f7876694099f4c35bdb417530e739328, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/84d59798828a4ed5bb290ca014fc7b6d, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/8bc490f73ef84c8f9bf0f7b655eb9873_SeqId_44_] to archive 2023-11-10 04:54:05,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:05,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/cd8136a290454513bb4d31935fa6b324 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/cd8136a290454513bb4d31935fa6b324 2023-11-10 04:54:05,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/d9ed9534448542b28b02f78e8f960921 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/d9ed9534448542b28b02f78e8f960921 2023-11-10 04:54:05,413 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d92e8b1875f64c768af9ae82f3bab168 first=Optional[r1] last=Optional[r2] 2023-11-10 04:54:05,413 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d92e8b1875f64c768af9ae82f3bab168}] 2023-11-10 04:54:05,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/57f86f085b6e42c68dd572881e2531fe to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/57f86f085b6e42c68dd572881e2531fe 2023-11-10 04:54:05,416 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/9400b3f4cab240e58d344795e691dd87 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/9400b3f4cab240e58d344795e691dd87 2023-11-10 04:54:05,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d92e8b1875f64c768af9ae82f3bab168 for inclusion in 422f46c0e9462ab44880fe4aeb8fb7a2/family1 2023-11-10 04:54:05,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/c01b79f89a48422ea1a1b01de601d41c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/c01b79f89a48422ea1a1b01de601d41c 2023-11-10 04:54:05,419 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/4682b8d048d74ba0982caa3ad0f41de2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/4682b8d048d74ba0982caa3ad0f41de2 2023-11-10 04:54:05,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-11-10 04:54:05,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:05,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:54:05,420 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/475e38a9c06f4fa484f922f1ec558cf7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/475e38a9c06f4fa484f922f1ec558cf7 2023-11-10 04:54:05,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/d92e8b1875f64c768af9ae82f3bab168 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9jjsajkkjaeihgsllv0sh49jtaiqrc5p1k3nfshocuobtbgckt3g4frnmkihrmc/family1/d92e8b1875f64c768af9ae82f3bab168 2023-11-10 04:54:05,422 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/f7876694099f4c35bdb417530e739328 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/f7876694099f4c35bdb417530e739328 2023-11-10 04:54:05,424 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/84d59798828a4ed5bb290ca014fc7b6d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/84d59798828a4ed5bb290ca014fc7b6d 2023-11-10 04:54:05,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9jjsajkkjaeihgsllv0sh49jtaiqrc5p1k3nfshocuobtbgckt3g4frnmkihrmc/family1/d92e8b1875f64c768af9ae82f3bab168 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/fae9e1efaab44ad2aeb26f4d54b59bd1_SeqId_47_ 2023-11-10 04:54:05,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9jjsajkkjaeihgsllv0sh49jtaiqrc5p1k3nfshocuobtbgckt3g4frnmkihrmc/family1/d92e8b1875f64c768af9ae82f3bab168 into 422f46c0e9462ab44880fe4aeb8fb7a2/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/fae9e1efaab44ad2aeb26f4d54b59bd1_SeqId_47_ - updating store file list. 2023-11-10 04:54:05,425 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/8bc490f73ef84c8f9bf0f7b655eb9873_SeqId_44_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/8bc490f73ef84c8f9bf0f7b655eb9873_SeqId_44_ 2023-11-10 04:54:05,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/fae9e1efaab44ad2aeb26f4d54b59bd1_SeqId_47_ into 422f46c0e9462ab44880fe4aeb8fb7a2/family1 2023-11-10 04:54:05,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9jjsajkkjaeihgsllv0sh49jtaiqrc5p1k3nfshocuobtbgckt3g4frnmkihrmc/family1/d92e8b1875f64c768af9ae82f3bab168 into 422f46c0e9462ab44880fe4aeb8fb7a2/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/fae9e1efaab44ad2aeb26f4d54b59bd1_SeqId_47_) 2023-11-10 04:54:05,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9jjsajkkjaeihgsllv0sh49jtaiqrc5p1k3nfshocuobtbgckt3g4frnmkihrmc/family1/d92e8b1875f64c768af9ae82f3bab168 2023-11-10 04:54:05,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-10 04:54:05,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-10 04:54:05,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 422f46c0e9462ab44880fe4aeb8fb7a2/family1 is initiating minor compaction (all files) 2023-11-10 04:54:05,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@11ea6ebc; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:05,431 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 422f46c0e9462ab44880fe4aeb8fb7a2/family1 in testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:54:05,431 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/a386c16b63b64cb1b4d1d1acb3d4a911, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/ec00424ab53d4e5dacde85981c354372, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/fae9e1efaab44ad2aeb26f4d54b59bd1_SeqId_47_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp, totalSize=176.8 K 2023-11-10 04:54:05,432 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting a386c16b63b64cb1b4d1d1acb3d4a911, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878827283 2023-11-10 04:54:05,432 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting ec00424ab53d4e5dacde85981c354372, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621227283 2023-11-10 04:54:05,432 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting fae9e1efaab44ad2aeb26f4d54b59bd1_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965227283 2023-11-10 04:54:05,433 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:05,434 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:05,436 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201e1e6ccaf3d9340a79677efde0d63fb88 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201e1e6ccaf3d9340a79677efde0d63fb88 2023-11-10 04:54:05,438 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205ff91b81ffbee461ea0e6886f4eed1475 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205ff91b81ffbee461ea0e6886f4eed1475 2023-11-10 04:54:05,438 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-11-10 04:54:05,438 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-10 04:54:05,438 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:05,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:05,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:05,440 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:54:05,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.4350 sec 2023-11-10 04:54:05,452 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 422f46c0e9462ab44880fe4aeb8fb7a2#family1#compaction#35 average throughput is 19.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:54:05,543 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=257 (was 259), OpenFileDescriptor=640 (was 645), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=224 (was 278), ProcessCount=170 (was 170), AvailableMemoryMB=10910 (was 11103) 2023-11-10 04:54:05,554 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=257, OpenFileDescriptor=640, MaxFileDescriptor=60000, SystemLoadAverage=224, ProcessCount=170, AvailableMemoryMB=10909 2023-11-10 04:54:05,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-10 04:54:05,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:05,559 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:54:05,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-11-10 04:54:05,560 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:54:05,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-10 04:54:05,563 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:05,563 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:05,563 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:05,564 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915 empty. 2023-11-10 04:54:05,564 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48 empty. 2023-11-10 04:54:05,564 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca empty. 2023-11-10 04:54:05,564 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:05,564 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:05,564 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:05,565 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-11-10 04:54:05,576 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-11-10 04:54:05,578 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => cbac8c6d54c66bf2d99bd01264a1e1ca, NAME => 'testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:54:05,578 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => a5bb60bfaec2aed551dc765427373c48, NAME => 'testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:54:05,578 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => d4d1a475f6a44a306a5cc4d860a9f915, NAME => 'testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:54:05,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-10 04:54:05,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-10 04:54:05,872 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/.tmp/family1/601e5abce3bf4a9ba83b5d661fbab7f4 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/601e5abce3bf4a9ba83b5d661fbab7f4 2023-11-10 04:54:05,877 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 422f46c0e9462ab44880fe4aeb8fb7a2/family1 of 422f46c0e9462ab44880fe4aeb8fb7a2 into 601e5abce3bf4a9ba83b5d661fbab7f4(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:54:05,877 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:54:05,877 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1699592045431; duration=0sec 2023-11-10 04:54:05,878 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:05,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/a386c16b63b64cb1b4d1d1acb3d4a911, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/ec00424ab53d4e5dacde85981c354372, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/fae9e1efaab44ad2aeb26f4d54b59bd1_SeqId_47_] to archive 2023-11-10 04:54:05,899 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:05,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/a386c16b63b64cb1b4d1d1acb3d4a911 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/a386c16b63b64cb1b4d1d1acb3d4a911 2023-11-10 04:54:05,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/ec00424ab53d4e5dacde85981c354372 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/ec00424ab53d4e5dacde85981c354372 2023-11-10 04:54:05,903 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/fae9e1efaab44ad2aeb26f4d54b59bd1_SeqId_47_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/family1/fae9e1efaab44ad2aeb26f4d54b59bd1_SeqId_47_ 2023-11-10 04:54:05,989 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:05,989 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing cbac8c6d54c66bf2d99bd01264a1e1ca, disabling compactions & flushes 2023-11-10 04:54:05,989 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:05,989 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:05,989 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. after waiting 0 ms 2023-11-10 04:54:05,989 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:05,990 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:05,990 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for cbac8c6d54c66bf2d99bd01264a1e1ca: 2023-11-10 04:54:05,994 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:05,995 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing d4d1a475f6a44a306a5cc4d860a9f915, disabling compactions & flushes 2023-11-10 04:54:05,995 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:05,995 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:05,995 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. after waiting 0 ms 2023-11-10 04:54:05,995 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:05,995 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:05,995 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for d4d1a475f6a44a306a5cc4d860a9f915: 2023-11-10 04:54:05,999 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:05,999 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing a5bb60bfaec2aed551dc765427373c48, disabling compactions & flushes 2023-11-10 04:54:05,999 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:05,999 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:05,999 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. after waiting 0 ms 2023-11-10 04:54:05,999 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:05,999 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:05,999 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for a5bb60bfaec2aed551dc765427373c48: 2023-11-10 04:54:06,002 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:54:06,002 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1699592046002"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592046002"}]},"ts":"1699592046002"} 2023-11-10 04:54:06,003 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1699592046002"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592046002"}]},"ts":"1699592046002"} 2023-11-10 04:54:06,003 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1699592046002"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592046002"}]},"ts":"1699592046002"} 2023-11-10 04:54:06,005 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-10 04:54:06,006 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:54:06,006 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592046006"}]},"ts":"1699592046006"} 2023-11-10 04:54:06,008 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-11-10 04:54:06,011 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a5bb60bfaec2aed551dc765427373c48, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=d4d1a475f6a44a306a5cc4d860a9f915, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=cbac8c6d54c66bf2d99bd01264a1e1ca, ASSIGN}] 2023-11-10 04:54:06,012 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=d4d1a475f6a44a306a5cc4d860a9f915, ASSIGN 2023-11-10 04:54:06,012 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=cbac8c6d54c66bf2d99bd01264a1e1ca, ASSIGN 2023-11-10 04:54:06,013 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a5bb60bfaec2aed551dc765427373c48, ASSIGN 2023-11-10 04:54:06,013 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=d4d1a475f6a44a306a5cc4d860a9f915, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:54:06,013 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a5bb60bfaec2aed551dc765427373c48, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:54:06,013 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=cbac8c6d54c66bf2d99bd01264a1e1ca, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:54:06,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-10 04:54:06,165 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d4d1a475f6a44a306a5cc4d860a9f915, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:06,165 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=cbac8c6d54c66bf2d99bd01264a1e1ca, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:06,165 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=a5bb60bfaec2aed551dc765427373c48, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:06,165 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1699592046165"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592046165"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592046165"}]},"ts":"1699592046165"} 2023-11-10 04:54:06,166 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1699592046165"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592046165"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592046165"}]},"ts":"1699592046165"} 2023-11-10 04:54:06,166 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1699592046165"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592046165"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592046165"}]},"ts":"1699592046165"} 2023-11-10 04:54:06,167 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=31, ppid=29, state=RUNNABLE; OpenRegionProcedure d4d1a475f6a44a306a5cc4d860a9f915, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:06,168 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure cbac8c6d54c66bf2d99bd01264a1e1ca, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:06,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=33, ppid=28, state=RUNNABLE; OpenRegionProcedure a5bb60bfaec2aed551dc765427373c48, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:06,319 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:06,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:06,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cbac8c6d54c66bf2d99bd01264a1e1ca, NAME => 'testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca.', STARTKEY => '2', ENDKEY => ''} 2023-11-10 04:54:06,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:06,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:06,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:06,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:06,325 INFO [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:06,326 DEBUG [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1 2023-11-10 04:54:06,326 DEBUG [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1 2023-11-10 04:54:06,327 INFO [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cbac8c6d54c66bf2d99bd01264a1e1ca columnFamilyName family1 2023-11-10 04:54:06,328 INFO [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] regionserver.HStore(310): Store=cbac8c6d54c66bf2d99bd01264a1e1ca/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:06,328 INFO [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:06,330 DEBUG [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family2 2023-11-10 04:54:06,330 DEBUG [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family2 2023-11-10 04:54:06,330 INFO [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cbac8c6d54c66bf2d99bd01264a1e1ca columnFamilyName family2 2023-11-10 04:54:06,332 INFO [StoreOpener-cbac8c6d54c66bf2d99bd01264a1e1ca-1] regionserver.HStore(310): Store=cbac8c6d54c66bf2d99bd01264a1e1ca/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:06,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:06,333 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:06,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:54:06,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:54:06,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:54:06,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cbac8c6d54c66bf2d99bd01264a1e1ca; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70357662, jitterRate=0.04841086268424988}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:54:06,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cbac8c6d54c66bf2d99bd01264a1e1ca: 2023-11-10 04:54:06,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca., pid=32, masterSystemTime=1699592046319 2023-11-10 04:54:06,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:06,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:06,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:06,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d4d1a475f6a44a306a5cc4d860a9f915, NAME => 'testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915.', STARTKEY => '1', ENDKEY => '2'} 2023-11-10 04:54:06,342 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=cbac8c6d54c66bf2d99bd01264a1e1ca, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:06,342 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1699592046341"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592046341"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592046341"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592046341"}]},"ts":"1699592046341"} 2023-11-10 04:54:06,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:06,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:06,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:06,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:06,343 INFO [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:06,345 DEBUG [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915/family1 2023-11-10 04:54:06,345 DEBUG [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915/family1 2023-11-10 04:54:06,345 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=32, resume processing ppid=30 2023-11-10 04:54:06,345 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure cbac8c6d54c66bf2d99bd01264a1e1ca, server=jenkins-hbase4.apache.org,36419,1699592002485 in 175 msec 2023-11-10 04:54:06,345 INFO [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4d1a475f6a44a306a5cc4d860a9f915 columnFamilyName family1 2023-11-10 04:54:06,346 INFO [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] regionserver.HStore(310): Store=d4d1a475f6a44a306a5cc4d860a9f915/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:06,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=cbac8c6d54c66bf2d99bd01264a1e1ca, ASSIGN in 334 msec 2023-11-10 04:54:06,347 INFO [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:06,348 DEBUG [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915/family2 2023-11-10 04:54:06,348 DEBUG [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915/family2 2023-11-10 04:54:06,349 INFO [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4d1a475f6a44a306a5cc4d860a9f915 columnFamilyName family2 2023-11-10 04:54:06,350 INFO [StoreOpener-d4d1a475f6a44a306a5cc4d860a9f915-1] regionserver.HStore(310): Store=d4d1a475f6a44a306a5cc4d860a9f915/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:06,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:06,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:06,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:54:06,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:54:06,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:54:06,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d4d1a475f6a44a306a5cc4d860a9f915; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64212831, jitterRate=-0.04315425455570221}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:54:06,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d4d1a475f6a44a306a5cc4d860a9f915: 2023-11-10 04:54:06,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915., pid=31, masterSystemTime=1699592046319 2023-11-10 04:54:06,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:06,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:06,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:06,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a5bb60bfaec2aed551dc765427373c48, NAME => 'testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48.', STARTKEY => '', ENDKEY => '1'} 2023-11-10 04:54:06,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:06,360 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=d4d1a475f6a44a306a5cc4d860a9f915, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:06,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:06,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:06,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:06,360 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1699592046360"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592046360"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592046360"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592046360"}]},"ts":"1699592046360"} 2023-11-10 04:54:06,361 INFO [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:06,363 DEBUG [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48/family1 2023-11-10 04:54:06,363 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=31, resume processing ppid=29 2023-11-10 04:54:06,363 DEBUG [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48/family1 2023-11-10 04:54:06,363 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=31, ppid=29, state=SUCCESS; OpenRegionProcedure d4d1a475f6a44a306a5cc4d860a9f915, server=jenkins-hbase4.apache.org,36419,1699592002485 in 195 msec 2023-11-10 04:54:06,364 INFO [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5bb60bfaec2aed551dc765427373c48 columnFamilyName family1 2023-11-10 04:54:06,365 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=d4d1a475f6a44a306a5cc4d860a9f915, ASSIGN in 352 msec 2023-11-10 04:54:06,365 INFO [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] regionserver.HStore(310): Store=a5bb60bfaec2aed551dc765427373c48/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:06,365 INFO [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:06,367 DEBUG [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48/family2 2023-11-10 04:54:06,367 DEBUG [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48/family2 2023-11-10 04:54:06,367 INFO [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a5bb60bfaec2aed551dc765427373c48 columnFamilyName family2 2023-11-10 04:54:06,368 INFO [StoreOpener-a5bb60bfaec2aed551dc765427373c48-1] regionserver.HStore(310): Store=a5bb60bfaec2aed551dc765427373c48/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:06,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:06,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:06,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:54:06,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:54:06,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:54:06,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a5bb60bfaec2aed551dc765427373c48; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63646020, jitterRate=-0.05160039663314819}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:54:06,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a5bb60bfaec2aed551dc765427373c48: 2023-11-10 04:54:06,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48., pid=33, masterSystemTime=1699592046319 2023-11-10 04:54:06,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:06,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:06,379 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=a5bb60bfaec2aed551dc765427373c48, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:06,379 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1699592046379"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592046379"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592046379"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592046379"}]},"ts":"1699592046379"} 2023-11-10 04:54:06,382 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=33, resume processing ppid=28 2023-11-10 04:54:06,382 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=33, ppid=28, state=SUCCESS; OpenRegionProcedure a5bb60bfaec2aed551dc765427373c48, server=jenkins-hbase4.apache.org,36419,1699592002485 in 211 msec 2023-11-10 04:54:06,384 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=28, resume processing ppid=27 2023-11-10 04:54:06,384 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=a5bb60bfaec2aed551dc765427373c48, ASSIGN in 371 msec 2023-11-10 04:54:06,385 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:54:06,385 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592046385"}]},"ts":"1699592046385"} 2023-11-10 04:54:06,386 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-11-10 04:54:06,390 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:54:06,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 833 msec 2023-11-10 04:54:06,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-10 04:54:06,666 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-11-10 04:54:06,668 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-11-10 04:54:06,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:06,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:06,676 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-11-10 04:54:06,676 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:06,676 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-11-10 04:54:06,676 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-11-10 04:54:06,677 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:06,677 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:06,679 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:06,679 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:06,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:06,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:06,679 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:06,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:06,679 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:06,679 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:06,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:06,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:06,680 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:06,680 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:06,680 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-11-10 04:54:06,680 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:06,680 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-11-10 04:54:06,681 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:06,681 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. started... 2023-11-10 04:54:06,681 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:54:06,681 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:06,681 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a5bb60bfaec2aed551dc765427373c48: 2023-11-10 04:54:06,681 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:06,682 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:06,681 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. started... 2023-11-10 04:54:06,681 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:06,682 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cbac8c6d54c66bf2d99bd01264a1e1ca 2/2 column families, dataSize=48 B heapSize=632 B 2023-11-10 04:54:06,682 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. started... 2023-11-10 04:54:06,682 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:54:06,683 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d4d1a475f6a44a306a5cc4d860a9f915: 2023-11-10 04:54:06,683 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:06,683 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:54:07,099 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311105fabf09956474406ab22ab378bbc84a9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202311105fabf09956474406ab22ab378bbc84a9 2023-11-10 04:54:07,505 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/.tmp/family1/a8ee8792a9c54a15970c33393229909b 2023-11-10 04:54:07,510 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/.tmp/family1/a8ee8792a9c54a15970c33393229909b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/a8ee8792a9c54a15970c33393229909b 2023-11-10 04:54:07,515 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/a8ee8792a9c54a15970c33393229909b, entries=1, sequenceid=5, filesize=4.9 K 2023-11-10 04:54:07,516 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for cbac8c6d54c66bf2d99bd01264a1e1ca in 834ms, sequenceid=5, compaction requested=false 2023-11-10 04:54:07,516 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cbac8c6d54c66bf2d99bd01264a1e1ca: 2023-11-10 04:54:07,516 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:07,517 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:54:07,517 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:54:07,517 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,517 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-11-10 04:54:07,517 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-11-10 04:54:07,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,518 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:07,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:07,519 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,519 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:07,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:07,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:07,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:07,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-11-10 04:54:07,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11a430e7[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:07,520 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-11-10 04:54:07,520 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,521 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,522 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-11-10 04:54:07,522 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-11-10 04:54:07,522 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:07,522 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,522 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:07,523 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,523 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-11-10 04:54:07,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:07,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:07,523 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:07,523 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-11-10 04:54:07,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:07,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:07,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:07,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:07,525 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-11-10 04:54:07,525 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-11-10 04:54:07,525 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:07,525 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:07,526 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,526 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:07,528 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,528 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:07,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,528 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:07,528 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,530 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:07,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,532 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,532 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:07,532 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:07,532 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:07,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-11-10 04:54:07,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:07,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:07,532 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,532 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:07,533 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,533 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:07,533 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,533 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:07,533 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,533 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:07,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:07,633 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:07,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:07,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:07,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:07,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,640 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-11-10 04:54:07,640 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:07,640 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-11-10 04:54:07,640 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-11-10 04:54:07,641 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,641 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,642 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,642 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:07,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:07,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:07,642 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:07,643 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:07,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:07,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,644 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,644 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-11-10 04:54:07,644 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:07,644 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-11-10 04:54:07,644 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:07,644 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-11-10 04:54:07,644 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:07,644 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. started... 2023-11-10 04:54:07,644 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:07,644 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:07,644 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:54:07,644 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:07,644 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. started... 2023-11-10 04:54:07,644 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a5bb60bfaec2aed551dc765427373c48: 2023-11-10 04:54:07,644 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. started... 2023-11-10 04:54:07,645 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cbac8c6d54c66bf2d99bd01264a1e1ca 2/2 column families, dataSize=90 B heapSize=752 B 2023-11-10 04:54:07,645 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:54:07,645 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d4d1a475f6a44a306a5cc4d860a9f915: 2023-11-10 04:54:07,645 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:54:07,645 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:54:07,645 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:54:08,058 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311109149a6f7ceed4d218217996e23732bc9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202311109149a6f7ceed4d218217996e23732bc9 2023-11-10 04:54:08,464 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/.tmp/family1/f0511b8d95664b5db80de46f0a74d155 2023-11-10 04:54:08,469 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/.tmp/family1/f0511b8d95664b5db80de46f0a74d155 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/f0511b8d95664b5db80de46f0a74d155 2023-11-10 04:54:08,474 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/f0511b8d95664b5db80de46f0a74d155, entries=2, sequenceid=9, filesize=5.0 K 2023-11-10 04:54:08,474 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for cbac8c6d54c66bf2d99bd01264a1e1ca in 829ms, sequenceid=9, compaction requested=false 2023-11-10 04:54:08,474 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cbac8c6d54c66bf2d99bd01264a1e1ca: 2023-11-10 04:54:08,474 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:08,475 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:54:08,475 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:54:08,475 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,475 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-11-10 04:54:08,475 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-11-10 04:54:08,477 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-11-10 04:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7db2137c[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:08,479 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-11-10 04:54:08,479 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,481 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,481 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-11-10 04:54:08,481 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-11-10 04:54:08,481 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:08,481 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,481 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:08,483 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,483 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-11-10 04:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:08,483 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:08,483 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-11-10 04:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:08,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-11-10 04:54:08,485 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-11-10 04:54:08,486 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:08,486 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:08,486 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,486 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:08,487 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,487 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,487 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:08,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:08,488 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:08,489 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:08,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,499 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,499 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,499 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:08,499 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,499 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:08,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:08,499 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:08,500 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:08,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:08,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-11-10 04:54:08,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:08,500 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,500 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:08,500 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:08,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:08,500 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,500 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-11-10 04:54:08,500 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:08,601 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:08,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:08,608 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-11-10 04:54:08,611 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-10 04:54:08,611 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-10 04:54:08,612 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592648612 2023-11-10 04:54:08,612 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:08,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:54:08,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-11-10 04:54:08,614 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-10 04:54:08,616 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x1142a68c to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:08,620 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@44c3f352, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:08,620 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231110 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-11-10 04:54:09,449 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110de7957065a09449ba4976ab32d26f3c2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231110de7957065a09449ba4976ab32d26f3c2 2023-11-10 04:54:09,450 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-10 04:54:09,463 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:54:09,464 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39730, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:54:09,471 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:54:09,473 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50078, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:54:09,482 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231110/family1/5fd7f6b8ef2b4b0c99593cd1b09dcdfd first=Optional[k0] last=Optional[k1] 2023-11-10 04:54:09,483 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231110/family1/5fd7f6b8ef2b4b0c99593cd1b09dcdfd}] 2023-11-10 04:54:09,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231110/family1/5fd7f6b8ef2b4b0c99593cd1b09dcdfd for inclusion in cbac8c6d54c66bf2d99bd01264a1e1ca/family1 2023-11-10 04:54:09,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-11-10 04:54:09,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-10 04:54:09,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for cbac8c6d54c66bf2d99bd01264a1e1ca: 2023-11-10 04:54:09,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231110/family1/5fd7f6b8ef2b4b0c99593cd1b09dcdfd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testScannerOnBulkLoadRefHFiles__rcmu85divj6ous545cldg616bqs1ehg3qulj0rtiej3gil9q09kbndnfn8gpokjf/family1/5fd7f6b8ef2b4b0c99593cd1b09dcdfd 2023-11-10 04:54:09,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testScannerOnBulkLoadRefHFiles__rcmu85divj6ous545cldg616bqs1ehg3qulj0rtiej3gil9q09kbndnfn8gpokjf/family1/5fd7f6b8ef2b4b0c99593cd1b09dcdfd as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/3079dc6b460749bfb8477985a522ffd2_SeqId_12_ 2023-11-10 04:54:09,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testScannerOnBulkLoadRefHFiles__rcmu85divj6ous545cldg616bqs1ehg3qulj0rtiej3gil9q09kbndnfn8gpokjf/family1/5fd7f6b8ef2b4b0c99593cd1b09dcdfd into cbac8c6d54c66bf2d99bd01264a1e1ca/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/3079dc6b460749bfb8477985a522ffd2_SeqId_12_ - updating store file list. 2023-11-10 04:54:09,516 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:09,516 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1bc1c5ad to 127.0.0.1:51169 2023-11-10 04:54:09,517 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:09,518 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:09,518 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x00e894d1 to 127.0.0.1:51169 2023-11-10 04:54:09,518 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:09,518 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:09,518 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f030c5d to 127.0.0.1:51169 2023-11-10 04:54:09,518 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:09,519 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:09,519 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b04a755 to 127.0.0.1:51169 2023-11-10 04:54:09,519 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:09,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/3079dc6b460749bfb8477985a522ffd2_SeqId_12_ into cbac8c6d54c66bf2d99bd01264a1e1ca/family1 2023-11-10 04:54:09,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testScannerOnBulkLoadRefHFiles__rcmu85divj6ous545cldg616bqs1ehg3qulj0rtiej3gil9q09kbndnfn8gpokjf/family1/5fd7f6b8ef2b4b0c99593cd1b09dcdfd into cbac8c6d54c66bf2d99bd01264a1e1ca/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/3079dc6b460749bfb8477985a522ffd2_SeqId_12_) 2023-11-10 04:54:09,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testScannerOnBulkLoadRefHFiles__rcmu85divj6ous545cldg616bqs1ehg3qulj0rtiej3gil9q09kbndnfn8gpokjf/family1/5fd7f6b8ef2b4b0c99593cd1b09dcdfd 2023-11-10 04:54:09,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-10 04:54:09,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-10 04:54:09,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): cbac8c6d54c66bf2d99bd01264a1e1ca/family1 is initiating minor compaction (all files) 2023-11-10 04:54:09,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6e9fbd95; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:09,525 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cbac8c6d54c66bf2d99bd01264a1e1ca/family1 in testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:54:09,525 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/a8ee8792a9c54a15970c33393229909b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/f0511b8d95664b5db80de46f0a74d155, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/3079dc6b460749bfb8477985a522ffd2_SeqId_12_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/.tmp, totalSize=15.1 K 2023-11-10 04:54:09,525 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting a8ee8792a9c54a15970c33393229909b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592046667 2023-11-10 04:54:09,526 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting f0511b8d95664b5db80de46f0a74d155, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592046667 2023-11-10 04:54:09,526 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 3079dc6b460749bfb8477985a522ffd2_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592046667 2023-11-10 04:54:09,527 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-10 04:54:09,528 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:09,529 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202311105fabf09956474406ab22ab378bbc84a9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202311105fabf09956474406ab22ab378bbc84a9 2023-11-10 04:54:09,530 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202311109149a6f7ceed4d218217996e23732bc9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202311109149a6f7ceed4d218217996e23732bc9 2023-11-10 04:54:09,531 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-11-10 04:54:09,531 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231110' 2023-11-10 04:54:09,531 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231110 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-11-10 04:54:09,531 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-10 04:54:09,531 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:09,532 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-10 04:54:09,534 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 922 msec 2023-11-10 04:54:09,537 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): cbac8c6d54c66bf2d99bd01264a1e1ca#family1#compaction#38 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:54:09,564 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=261 (was 257) - Thread LEAK? -, OpenFileDescriptor=626 (was 640), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=224 (was 224), ProcessCount=170 (was 170), AvailableMemoryMB=10837 (was 10909) 2023-11-10 04:54:09,575 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=261, OpenFileDescriptor=626, MaxFileDescriptor=60000, SystemLoadAverage=224, ProcessCount=170, AvailableMemoryMB=10837 2023-11-10 04:54:09,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-10 04:54:09,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:09,579 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:54:09,579 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-11-10 04:54:09,580 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:54:09,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-10 04:54:09,581 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:54:09,582 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29 empty. 2023-11-10 04:54:09,582 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:54:09,582 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-11-10 04:54:09,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-10 04:54:09,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-10 04:54:09,990 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-10 04:54:09,991 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 92e368b193b9f67ef8fc1cb88c465c29, NAME => 'testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:54:10,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-10 04:54:10,397 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:10,397 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 92e368b193b9f67ef8fc1cb88c465c29, disabling compactions & flushes 2023-11-10 04:54:10,397 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:10,397 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:10,397 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. after waiting 0 ms 2023-11-10 04:54:10,397 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:10,397 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:10,397 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:10,400 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:54:10,401 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1699592050401"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592050401"}]},"ts":"1699592050401"} 2023-11-10 04:54:10,402 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-10 04:54:10,403 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:54:10,403 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592050403"}]},"ts":"1699592050403"} 2023-11-10 04:54:10,404 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-11-10 04:54:10,408 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=92e368b193b9f67ef8fc1cb88c465c29, ASSIGN}] 2023-11-10 04:54:10,409 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=92e368b193b9f67ef8fc1cb88c465c29, ASSIGN 2023-11-10 04:54:10,410 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=92e368b193b9f67ef8fc1cb88c465c29, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:54:10,561 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=92e368b193b9f67ef8fc1cb88c465c29, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:10,561 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1699592050561"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592050561"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592050561"}]},"ts":"1699592050561"} 2023-11-10 04:54:10,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 92e368b193b9f67ef8fc1cb88c465c29, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:10,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-10 04:54:10,715 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:10,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:10,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 92e368b193b9f67ef8fc1cb88c465c29, NAME => 'testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:54:10,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:54:10,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:10,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:54:10,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:54:10,720 INFO [StoreOpener-92e368b193b9f67ef8fc1cb88c465c29-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:54:10,721 DEBUG [StoreOpener-92e368b193b9f67ef8fc1cb88c465c29-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1 2023-11-10 04:54:10,722 DEBUG [StoreOpener-92e368b193b9f67ef8fc1cb88c465c29-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1 2023-11-10 04:54:10,722 INFO [StoreOpener-92e368b193b9f67ef8fc1cb88c465c29-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 92e368b193b9f67ef8fc1cb88c465c29 columnFamilyName family1 2023-11-10 04:54:10,723 INFO [StoreOpener-92e368b193b9f67ef8fc1cb88c465c29-1] regionserver.HStore(310): Store=92e368b193b9f67ef8fc1cb88c465c29/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:10,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:54:10,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:54:10,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:54:10,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:54:10,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 92e368b193b9f67ef8fc1cb88c465c29; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63291331, jitterRate=-0.056885674595832825}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:54:10,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:10,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29., pid=37, masterSystemTime=1699592050715 2023-11-10 04:54:10,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:10,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:10,733 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=92e368b193b9f67ef8fc1cb88c465c29, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:10,733 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1699592050733"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592050733"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592050733"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592050733"}]},"ts":"1699592050733"} 2023-11-10 04:54:10,736 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=37, resume processing ppid=36 2023-11-10 04:54:10,736 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 92e368b193b9f67ef8fc1cb88c465c29, server=jenkins-hbase4.apache.org,36419,1699592002485 in 171 msec 2023-11-10 04:54:10,738 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=36, resume processing ppid=35 2023-11-10 04:54:10,738 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=92e368b193b9f67ef8fc1cb88c465c29, ASSIGN in 328 msec 2023-11-10 04:54:10,738 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:54:10,738 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592050738"}]},"ts":"1699592050738"} 2023-11-10 04:54:10,739 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-11-10 04:54:10,741 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:54:10,743 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1640 sec 2023-11-10 04:54:10,760 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/.tmp/family1/fdf74c33e40244d4a53c050a9236e559 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/fdf74c33e40244d4a53c050a9236e559 2023-11-10 04:54:10,765 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in cbac8c6d54c66bf2d99bd01264a1e1ca/family1 of cbac8c6d54c66bf2d99bd01264a1e1ca into fdf74c33e40244d4a53c050a9236e559(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:54:10,765 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cbac8c6d54c66bf2d99bd01264a1e1ca: 2023-11-10 04:54:10,765 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1699592049525; duration=1sec 2023-11-10 04:54:10,765 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:10,805 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/a8ee8792a9c54a15970c33393229909b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/f0511b8d95664b5db80de46f0a74d155, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/3079dc6b460749bfb8477985a522ffd2_SeqId_12_] to archive 2023-11-10 04:54:10,806 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:10,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/a8ee8792a9c54a15970c33393229909b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/a8ee8792a9c54a15970c33393229909b 2023-11-10 04:54:10,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/f0511b8d95664b5db80de46f0a74d155 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/f0511b8d95664b5db80de46f0a74d155 2023-11-10 04:54:10,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/3079dc6b460749bfb8477985a522ffd2_SeqId_12_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/family1/3079dc6b460749bfb8477985a522ffd2_SeqId_12_ 2023-11-10 04:54:11,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-10 04:54:11,685 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-11-10 04:54:11,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:11,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:11,708 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:11,708 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:11,708 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:11,708 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:11,709 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:11,709 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:11,710 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:11,710 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:11,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:11,711 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:11,711 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:11,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:11,711 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:11,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:11,712 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:11,712 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:11,712 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:11,712 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:11,712 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:11,712 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:11,712 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:11,712 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-10 04:54:11,730 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f9720a9084b84c36865b74cf30824246 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130f9720a9084b84c36865b74cf30824246 2023-11-10 04:54:11,851 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:54:12,134 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/45d6f5b3bb61477c8d7c0591930719b9 2023-11-10 04:54:12,140 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/45d6f5b3bb61477c8d7c0591930719b9 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/45d6f5b3bb61477c8d7c0591930719b9 2023-11-10 04:54:12,145 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/45d6f5b3bb61477c8d7c0591930719b9, entries=1000, sequenceid=5, filesize=165.1 K 2023-11-10 04:54:12,146 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 434ms, sequenceid=5, compaction requested=false 2023-11-10 04:54:12,146 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:12,146 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:12,146 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:12,146 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:12,147 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:12,147 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,147 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:12,147 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:12,149 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,149 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:12,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:12,150 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,150 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:12,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:12,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6be53577[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:12,152 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:12,152 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,153 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,153 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:12,153 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:12,153 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:12,153 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,153 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:12,154 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:12,154 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,154 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:12,155 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:12,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:12,157 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:12,157 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:12,157 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:12,157 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,157 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:12,159 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,159 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:12,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:12,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,159 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:12,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:12,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:12,159 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:12,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:12,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,161 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:12,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,169 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,169 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,170 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,169 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:12,170 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,170 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:12,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:12,170 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,170 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,170 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:12,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-10 04:54:12,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:12,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:12,170 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:12,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:12,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:12,170 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:12,170 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:12,270 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:12,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:12,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:12,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:12,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,277 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:12,277 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:12,277 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:12,277 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:12,278 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,278 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,279 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,279 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:12,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:12,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:12,279 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:12,279 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:12,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:12,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,280 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:12,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:12,281 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:12,281 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:12,281 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:12,281 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:12,281 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:12,281 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:12,281 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:12,281 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-10 04:54:12,696 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112858110ed6e0a5462c9189054ef44cd94e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112858110ed6e0a5462c9189054ef44cd94e 2023-11-10 04:54:13,102 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/5b62c248bc934408a7dfbbf65e02f0c2 2023-11-10 04:54:13,107 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/5b62c248bc934408a7dfbbf65e02f0c2 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5b62c248bc934408a7dfbbf65e02f0c2 2023-11-10 04:54:13,112 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5b62c248bc934408a7dfbbf65e02f0c2, entries=1, sequenceid=9, filesize=5.0 K 2023-11-10 04:54:13,112 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 831ms, sequenceid=9, compaction requested=false 2023-11-10 04:54:13,113 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:13,113 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:13,113 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:13,113 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:13,113 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,113 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:13,113 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:13,114 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,114 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,115 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:13,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:13,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:13,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54fc5222[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:13,117 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:13,117 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,118 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,118 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:13,118 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:13,118 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:13,118 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,118 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:13,121 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,121 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:13,121 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:13,121 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:13,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:13,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:13,123 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:13,123 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:13,123 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:13,123 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:13,123 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,123 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:13,125 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,125 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:13,125 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:13,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:13,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:13,125 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:13,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,127 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:13,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:13,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:13,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:13,132 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:13,132 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:13,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:13,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:13,233 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:13,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:13,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:13,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:13,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,239 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:13,239 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:13,239 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:13,239 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:13,240 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,240 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,242 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:13,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:13,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:13,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:13,243 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,243 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:13,243 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:13,244 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:13,244 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:13,244 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:13,244 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:13,244 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:13,244 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:13,244 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:13,244 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:13,656 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512013f5a12ecbbcc4813b67316f6e021b233 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512013f5a12ecbbcc4813b67316f6e021b233 2023-11-10 04:54:14,062 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/03e32a99f0bb43ffa60a52ba08e12470 2023-11-10 04:54:14,067 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/03e32a99f0bb43ffa60a52ba08e12470 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/03e32a99f0bb43ffa60a52ba08e12470 2023-11-10 04:54:14,071 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/03e32a99f0bb43ffa60a52ba08e12470, entries=1, sequenceid=13, filesize=4.9 K 2023-11-10 04:54:14,072 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 828ms, sequenceid=13, compaction requested=true 2023-11-10 04:54:14,072 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:14,072 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:14,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:14,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:14,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:14,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:14,074 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,074 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:14,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:14,074 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,074 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:14,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:14,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:14,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:14,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:14,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@646f6352[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:14,075 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:14,075 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,076 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,077 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:14,077 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:14,077 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:14,077 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,077 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:14,078 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:14,078 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:14,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:14,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:14,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:14,080 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:14,081 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:14,081 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:14,081 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,081 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:14,083 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,083 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:14,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,083 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:14,083 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:14,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:14,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,084 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:14,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:14,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-10 04:54:14,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:14,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:14,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:14,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:14,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:14,087 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:14,088 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:14,188 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:14,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:14,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:14,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:14,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,194 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:14,194 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:14,194 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:14,194 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:14,195 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,195 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,196 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:14,196 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:14,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:14,196 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:14,196 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:14,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:14,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:14,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:14,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:14,198 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:14,198 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:14,198 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:14,610 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205317c0bcf959343e4b1993aaa1ca63433 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205317c0bcf959343e4b1993aaa1ca63433 2023-11-10 04:54:15,016 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/2cbae9d3d4d94e25b0279478ef43690f 2023-11-10 04:54:15,021 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/2cbae9d3d4d94e25b0279478ef43690f as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/2cbae9d3d4d94e25b0279478ef43690f 2023-11-10 04:54:15,025 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/2cbae9d3d4d94e25b0279478ef43690f, entries=1, sequenceid=17, filesize=4.9 K 2023-11-10 04:54:15,026 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 828ms, sequenceid=17, compaction requested=true 2023-11-10 04:54:15,026 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:15,026 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:15,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:15,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:15,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:15,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:15,027 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,028 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:15,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:15,028 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:15,028 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:15,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:15,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:15,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:15,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@92bb550[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:15,029 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:15,029 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,031 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,032 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:15,032 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:15,032 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:15,032 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,032 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:15,033 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,033 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:15,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:15,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:15,033 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:15,033 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:15,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:15,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:15,035 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:15,035 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:15,035 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:15,035 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,035 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:15,037 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,037 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:15,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:15,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,037 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:15,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:15,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:15,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:15,037 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:15,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,039 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:15,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,041 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,041 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,041 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:15,041 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:15,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:15,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:15,042 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:15,041 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-10 04:54:15,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:15,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:15,042 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:15,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:15,042 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,042 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,042 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,042 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:15,042 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:15,142 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:15,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:15,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:15,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:15,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,149 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:15,149 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:15,149 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:15,149 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:15,150 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,150 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,153 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,153 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:15,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:15,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:15,153 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,153 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:15,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:15,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,154 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,154 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:15,154 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,154 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:15,154 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:15,154 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:15,154 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:15,155 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:15,155 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:15,155 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:15,155 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:15,567 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122876f3b219531547e9b15b0bafff4f627c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122876f3b219531547e9b15b0bafff4f627c 2023-11-10 04:54:15,973 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/5bb8cbb7b1ca40889ff0f8d1dd2c5dac 2023-11-10 04:54:15,978 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/5bb8cbb7b1ca40889ff0f8d1dd2c5dac as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5bb8cbb7b1ca40889ff0f8d1dd2c5dac 2023-11-10 04:54:15,983 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5bb8cbb7b1ca40889ff0f8d1dd2c5dac, entries=1, sequenceid=21, filesize=4.9 K 2023-11-10 04:54:15,984 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 829ms, sequenceid=21, compaction requested=true 2023-11-10 04:54:15,984 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:15,984 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:15,984 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:15,984 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:15,984 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,984 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:15,984 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:15,986 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,986 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:15,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:15,986 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,986 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:15,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:15,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ba7778f[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:15,988 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:15,988 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,989 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,989 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:15,989 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:15,989 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:15,989 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,989 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:15,991 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:15,991 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:15,991 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,991 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:15,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:15,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:15,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:15,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:15,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:15,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:15,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:15,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:15,993 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:15,993 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:15,993 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:15,993 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:15,993 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:16,000 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,000 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,000 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:16,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:16,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:16,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:16,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:16,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:16,000 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:16,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,002 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,004 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,004 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,004 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:16,004 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,004 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:16,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:16,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:16,004 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:16,004 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:16,004 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,004 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:16,004 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,005 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,005 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:16,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:16,005 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:16,005 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:16,105 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:16,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:16,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:16,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:16,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,112 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:16,112 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:16,112 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:16,112 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:16,113 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,113 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,114 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:16,114 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:16,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:16,114 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,114 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:16,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:16,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,116 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:16,116 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:16,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:16,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:16,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:16,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:16,116 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:16,116 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:16,116 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:16,529 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123118267e8b97374a5e93e3e2d3ecc24f52 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123118267e8b97374a5e93e3e2d3ecc24f52 2023-11-10 04:54:16,935 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/98bfc4536bba48dd99cdaf6ed717f09b 2023-11-10 04:54:16,940 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/98bfc4536bba48dd99cdaf6ed717f09b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/98bfc4536bba48dd99cdaf6ed717f09b 2023-11-10 04:54:16,945 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/98bfc4536bba48dd99cdaf6ed717f09b, entries=1, sequenceid=25, filesize=4.9 K 2023-11-10 04:54:16,945 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 829ms, sequenceid=25, compaction requested=true 2023-11-10 04:54:16,946 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:16,946 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:16,946 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:16,947 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,947 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:16,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:16,948 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,948 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:16,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:16,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:16,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:16,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7dfc419b[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:16,949 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:16,949 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,951 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,951 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:16,952 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:16,952 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:16,952 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,952 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:16,953 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,953 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,953 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:16,953 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:16,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:16,955 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:16,955 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:16,955 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:16,955 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,955 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:16,957 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,957 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:16,957 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:16,958 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:16,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,959 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:16,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,961 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,961 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,961 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,961 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:16,961 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,961 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:16,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:16,961 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:16,961 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:16,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-10 04:54:16,961 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:16,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:16,962 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:16,961 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:16,962 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:16,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:16,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:16,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:17,062 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:17,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:17,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:17,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:17,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,069 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:17,069 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:17,069 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:17,069 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:17,069 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,069 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,071 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:17,071 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:17,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:17,072 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,072 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:17,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:17,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,073 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,073 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:17,073 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,073 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:17,073 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:17,073 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:17,073 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:17,073 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:17,073 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:17,073 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:17,073 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:17,486 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101971c14fc46e847968c6044c9b685bc31 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101971c14fc46e847968c6044c9b685bc31 2023-11-10 04:54:17,891 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/80cbb1f50a0549769178ef759135fbe6 2023-11-10 04:54:17,896 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/80cbb1f50a0549769178ef759135fbe6 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/80cbb1f50a0549769178ef759135fbe6 2023-11-10 04:54:17,900 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/80cbb1f50a0549769178ef759135fbe6, entries=1, sequenceid=29, filesize=4.9 K 2023-11-10 04:54:17,901 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 828ms, sequenceid=29, compaction requested=true 2023-11-10 04:54:17,901 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:17,901 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:17,901 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:17,901 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:17,901 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,901 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:17,901 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:17,903 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,903 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:17,903 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,903 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:17,904 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:17,904 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6296d885[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:17,904 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:17,904 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,905 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,905 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:17,905 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:17,905 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:17,906 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,906 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:17,907 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,907 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:17,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:17,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:17,907 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:17,907 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:17,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:17,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:17,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:17,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:17,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:17,909 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:17,909 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:17,909 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:17,909 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,909 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:17,911 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,911 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:17,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:17,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,911 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:17,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:17,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:17,912 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:17,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:17,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,913 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:17,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:17,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,915 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:17,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-10 04:54:17,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:17,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:17,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:17,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:17,916 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,916 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:17,916 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:17,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:17,916 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:17,916 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:17,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:17,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:18,016 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:18,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:18,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:18,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:18,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,023 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:18,023 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:18,023 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:18,023 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:18,023 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,023 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,025 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:18,025 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:18,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:18,025 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,025 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:18,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:18,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,026 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,026 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:18,026 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:18,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:18,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:18,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:18,026 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:18,026 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:18,026 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:18,026 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:18,439 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010329aeb35ef3044c9ab12003840ae98892 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010329aeb35ef3044c9ab12003840ae98892 2023-11-10 04:54:18,844 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/6c1795ecbf7d4cb9b77cf06714fd2eb0 2023-11-10 04:54:18,849 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/6c1795ecbf7d4cb9b77cf06714fd2eb0 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/6c1795ecbf7d4cb9b77cf06714fd2eb0 2023-11-10 04:54:18,853 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/6c1795ecbf7d4cb9b77cf06714fd2eb0, entries=1, sequenceid=33, filesize=4.9 K 2023-11-10 04:54:18,854 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 828ms, sequenceid=33, compaction requested=true 2023-11-10 04:54:18,855 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:18,855 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:18,855 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:18,855 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:18,855 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,855 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:18,855 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:18,856 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,857 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:18,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:18,857 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,857 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:18,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:18,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:18,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:18,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:18,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@364370b2[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:18,858 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:18,858 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,867 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,867 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:18,867 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:18,867 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:18,867 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,867 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:18,868 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,868 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:18,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:18,868 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:18,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:18,868 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:18,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:18,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:18,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:18,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:18,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:18,871 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:18,871 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:18,871 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:18,871 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,871 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:18,872 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,872 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,872 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:18,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:18,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:18,873 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,874 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,876 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,876 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,876 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,876 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,876 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:18,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:18,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:18,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-10 04:54:18,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:18,877 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:18,876 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:18,876 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:18,877 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,877 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:18,877 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:18,877 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:18,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:18,977 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:18,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:18,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:18,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:18,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,984 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:18,984 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:18,984 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:18,984 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:18,984 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,984 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,989 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:18,989 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:18,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:18,990 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:18,990 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:18,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:18,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,990 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:18,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:18,991 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:18,991 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:18,991 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:18,991 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:18,991 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:18,991 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:18,991 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:18,991 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:19,406 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079b4ffbd535bf4a3b82bdd81b635f5e6f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201509079b4ffbd535bf4a3b82bdd81b635f5e6f 2023-11-10 04:54:19,811 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/27e4339fdc0f4fd2affbeadef164f79a 2023-11-10 04:54:19,816 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/27e4339fdc0f4fd2affbeadef164f79a as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/27e4339fdc0f4fd2affbeadef164f79a 2023-11-10 04:54:19,820 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/27e4339fdc0f4fd2affbeadef164f79a, entries=1, sequenceid=37, filesize=4.9 K 2023-11-10 04:54:19,821 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 830ms, sequenceid=37, compaction requested=true 2023-11-10 04:54:19,821 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:19,821 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:19,821 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:19,821 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:19,821 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,821 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:19,821 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:19,823 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,823 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:19,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:19,823 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,823 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:19,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:19,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:19,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:19,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:19,825 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:19,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46a872fa[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:19,825 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,826 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,826 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:19,826 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:19,826 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:19,826 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,826 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:19,827 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:19,828 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:19,828 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,828 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:19,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:19,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:19,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:19,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:19,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:19,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:19,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:19,830 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:19,830 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:19,830 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:19,830 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,830 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:19,832 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,832 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,832 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:19,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:19,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:19,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:19,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:19,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:19,833 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:19,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,834 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:19,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,836 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,836 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,836 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,836 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:19,837 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:19,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:19,837 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:19,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:19,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:19,837 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:19,837 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,837 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,837 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:19,837 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:19,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:19,837 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:19,937 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:19,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:19,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:19,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:19,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,944 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:19,944 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:19,944 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-10 04:54:19,944 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-10 04:54:19,944 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,945 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,946 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,946 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:19,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:19,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:19,946 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:19,946 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:19,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:19,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,947 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,947 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-10 04:54:19,947 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:19,947 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-10 04:54:19,948 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:19,948 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-10 04:54:19,948 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:19,948 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:19,948 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:19,948 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. started... 2023-11-10 04:54:19,948 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 92e368b193b9f67ef8fc1cb88c465c29 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:20,361 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120b9187dbd9e2345a989094f99c90b9960 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120b9187dbd9e2345a989094f99c90b9960 2023-11-10 04:54:20,767 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/bcaffe0d3b364ebdbfa8c8539dd97eef 2023-11-10 04:54:20,772 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/bcaffe0d3b364ebdbfa8c8539dd97eef as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/bcaffe0d3b364ebdbfa8c8539dd97eef 2023-11-10 04:54:20,777 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/bcaffe0d3b364ebdbfa8c8539dd97eef, entries=1, sequenceid=41, filesize=4.9 K 2023-11-10 04:54:20,778 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 92e368b193b9f67ef8fc1cb88c465c29 in 830ms, sequenceid=41, compaction requested=true 2023-11-10 04:54:20,778 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:20,778 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:20,778 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:20,778 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:20,778 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,778 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-10 04:54:20,778 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-10 04:54:20,780 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,780 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:20,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:20,780 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,780 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:20,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:20,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:20,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:20,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-10 04:54:20,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5947132d[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:20,782 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-10 04:54:20,782 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,784 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,784 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-10 04:54:20,784 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-10 04:54:20,784 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:20,784 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,784 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:20,785 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-10 04:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:20,786 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:20,786 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-10 04:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:20,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:20,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:54:20,788 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-10 04:54:20,788 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:20,788 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:20,788 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,788 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:20,789 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,789 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:20,790 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:20,790 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:20,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,792 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:20,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,798 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,798 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,798 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:20,798 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-10 04:54:20,799 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:20,798 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:20,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-10 04:54:20,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:20,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:20,798 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:20,799 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,799 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,799 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,799 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:20,799 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:20,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:20,899 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:20,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:20,901 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-11-10 04:54:20,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:54:20,904 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:54:20,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:20,905 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592660905 2023-11-10 04:54:20,907 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:54:20,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-11-10 04:54:20,908 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:20,910 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x43607900 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:20,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18624b5d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:20,915 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,357 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b32dff55e0a0414a9037a569fba85818 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103b32dff55e0a0414a9037a569fba85818 2023-11-10 04:54:21,359 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:21,375 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:54:21,376 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48998, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:54:21,382 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:54:21,383 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39148, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:54:21,392 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/28805a1c1c1542e693d6019cc80e09c5 first=Optional[r3] last=Optional[r6] 2023-11-10 04:54:21,392 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/28805a1c1c1542e693d6019cc80e09c5}] 2023-11-10 04:54:21,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/28805a1c1c1542e693d6019cc80e09c5 for inclusion in 92e368b193b9f67ef8fc1cb88c465c29/family1 2023-11-10 04:54:21,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-10 04:54:21,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:21,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:21,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/28805a1c1c1542e693d6019cc80e09c5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__gtipoieduh4pm98h522onase89b49qbm1s19rodvjmuhf8goa0927oe0lic0j0p7/family1/28805a1c1c1542e693d6019cc80e09c5 2023-11-10 04:54:21,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__gtipoieduh4pm98h522onase89b49qbm1s19rodvjmuhf8goa0927oe0lic0j0p7/family1/28805a1c1c1542e693d6019cc80e09c5 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/d468b5c4a9b540d9a371c223454cd953_SeqId_44_ 2023-11-10 04:54:21,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__gtipoieduh4pm98h522onase89b49qbm1s19rodvjmuhf8goa0927oe0lic0j0p7/family1/28805a1c1c1542e693d6019cc80e09c5 into 92e368b193b9f67ef8fc1cb88c465c29/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/d468b5c4a9b540d9a371c223454cd953_SeqId_44_ - updating store file list. 2023-11-10 04:54:21,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/d468b5c4a9b540d9a371c223454cd953_SeqId_44_ into 92e368b193b9f67ef8fc1cb88c465c29/family1 2023-11-10 04:54:21,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__gtipoieduh4pm98h522onase89b49qbm1s19rodvjmuhf8goa0927oe0lic0j0p7/family1/28805a1c1c1542e693d6019cc80e09c5 into 92e368b193b9f67ef8fc1cb88c465c29/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/d468b5c4a9b540d9a371c223454cd953_SeqId_44_) 2023-11-10 04:54:21,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__gtipoieduh4pm98h522onase89b49qbm1s19rodvjmuhf8goa0927oe0lic0j0p7/family1/28805a1c1c1542e693d6019cc80e09c5 2023-11-10 04:54:21,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-10 04:54:21,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-10 04:54:21,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 92e368b193b9f67ef8fc1cb88c465c29/family1 is initiating minor compaction 2023-11-10 04:54:21,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6a479108; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:21,411 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 92e368b193b9f67ef8fc1cb88c465c29/family1 in testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:54:21,411 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5b62c248bc934408a7dfbbf65e02f0c2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/03e32a99f0bb43ffa60a52ba08e12470, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/2cbae9d3d4d94e25b0279478ef43690f, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5bb8cbb7b1ca40889ff0f8d1dd2c5dac, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/98bfc4536bba48dd99cdaf6ed717f09b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/80cbb1f50a0549769178ef759135fbe6, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/6c1795ecbf7d4cb9b77cf06714fd2eb0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/27e4339fdc0f4fd2affbeadef164f79a, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/bcaffe0d3b364ebdbfa8c8539dd97eef, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/d468b5c4a9b540d9a371c223454cd953_SeqId_44_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp, totalSize=50.0 K 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 5b62c248bc934408a7dfbbf65e02f0c2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 03e32a99f0bb43ffa60a52ba08e12470, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 2cbae9d3d4d94e25b0279478ef43690f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 5bb8cbb7b1ca40889ff0f8d1dd2c5dac, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 98bfc4536bba48dd99cdaf6ed717f09b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 80cbb1f50a0549769178ef759135fbe6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 6c1795ecbf7d4cb9b77cf06714fd2eb0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 27e4339fdc0f4fd2affbeadef164f79a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting bcaffe0d3b364ebdbfa8c8539dd97eef, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-10 04:54:21,412 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting d468b5c4a9b540d9a371c223454cd953_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-10 04:54:21,414 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:21,415 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:21,417 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122876f3b219531547e9b15b0bafff4f627c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122876f3b219531547e9b15b0bafff4f627c 2023-11-10 04:54:21,418 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123118267e8b97374a5e93e3e2d3ecc24f52 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123118267e8b97374a5e93e3e2d3ecc24f52 2023-11-10 04:54:21,420 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101971c14fc46e847968c6044c9b685bc31 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101971c14fc46e847968c6044c9b685bc31 2023-11-10 04:54:21,421 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010329aeb35ef3044c9ab12003840ae98892 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010329aeb35ef3044c9ab12003840ae98892 2023-11-10 04:54:21,421 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-11-10 04:54:21,421 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-10 04:54:21,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,451 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 92e368b193b9f67ef8fc1cb88c465c29#family1#compaction#49 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:54:21,479 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512058337914a53c94dc38894237661bae821 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512058337914a53c94dc38894237661bae821 2023-11-10 04:54:21,480 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:21,512 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f44a11b8dcc44dcbb1cc41caca63687 first=Optional[r00] last=Optional[r2] 2023-11-10 04:54:21,513 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f44a11b8dcc44dcbb1cc41caca63687}] 2023-11-10 04:54:21,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f44a11b8dcc44dcbb1cc41caca63687 for inclusion in 92e368b193b9f67ef8fc1cb88c465c29/family1 2023-11-10 04:54:21,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-10 04:54:21,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:21,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:21,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f44a11b8dcc44dcbb1cc41caca63687 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__cbv4pe182dklu9qopmkf5v7jh323rffd9aedmm6tihndsmojimor2ffvmqoc7k2f/family1/7f44a11b8dcc44dcbb1cc41caca63687 2023-11-10 04:54:21,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__cbv4pe182dklu9qopmkf5v7jh323rffd9aedmm6tihndsmojimor2ffvmqoc7k2f/family1/7f44a11b8dcc44dcbb1cc41caca63687 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/c4a27ce1193a4b87ad3253784d521210_SeqId_46_ 2023-11-10 04:54:21,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__cbv4pe182dklu9qopmkf5v7jh323rffd9aedmm6tihndsmojimor2ffvmqoc7k2f/family1/7f44a11b8dcc44dcbb1cc41caca63687 into 92e368b193b9f67ef8fc1cb88c465c29/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/c4a27ce1193a4b87ad3253784d521210_SeqId_46_ - updating store file list. 2023-11-10 04:54:21,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/c4a27ce1193a4b87ad3253784d521210_SeqId_46_ into 92e368b193b9f67ef8fc1cb88c465c29/family1 2023-11-10 04:54:21,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__cbv4pe182dklu9qopmkf5v7jh323rffd9aedmm6tihndsmojimor2ffvmqoc7k2f/family1/7f44a11b8dcc44dcbb1cc41caca63687 into 92e368b193b9f67ef8fc1cb88c465c29/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/c4a27ce1193a4b87ad3253784d521210_SeqId_46_) 2023-11-10 04:54:21,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyPolicy__cbv4pe182dklu9qopmkf5v7jh323rffd9aedmm6tihndsmojimor2ffvmqoc7k2f/family1/7f44a11b8dcc44dcbb1cc41caca63687 2023-11-10 04:54:21,526 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:21,527 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:21,528 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130f9720a9084b84c36865b74cf30824246 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130f9720a9084b84c36865b74cf30824246 2023-11-10 04:54:21,530 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512013f5a12ecbbcc4813b67316f6e021b233 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512013f5a12ecbbcc4813b67316f6e021b233 2023-11-10 04:54:21,530 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205317c0bcf959343e4b1993aaa1ca63433 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205317c0bcf959343e4b1993aaa1ca63433 2023-11-10 04:54:21,531 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-11-10 04:54:21,531 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-10 04:54:21,531 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-10 04:54:21,531 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-10 04:54:21,531 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:21,532 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:54:21,533 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 630 msec 2023-11-10 04:54:21,637 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=257 (was 261), OpenFileDescriptor=656 (was 626) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=183 (was 224), ProcessCount=170 (was 170), AvailableMemoryMB=10832 (was 10837) 2023-11-10 04:54:21,650 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=257, OpenFileDescriptor=656, MaxFileDescriptor=60000, SystemLoadAverage=183, ProcessCount=170, AvailableMemoryMB=10831 2023-11-10 04:54:21,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-10 04:54:21,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:21,654 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:54:21,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-11-10 04:54:21,654 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:54:21,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-10 04:54:21,656 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:21,656 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3 empty. 2023-11-10 04:54:21,657 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:21,657 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-11-10 04:54:21,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-10 04:54:21,875 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/.tmp/family1/c8d80d863cad4c2585756c911d6ffb6a as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/c8d80d863cad4c2585756c911d6ffb6a 2023-11-10 04:54:21,880 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 92e368b193b9f67ef8fc1cb88c465c29/family1 of 92e368b193b9f67ef8fc1cb88c465c29 into c8d80d863cad4c2585756c911d6ffb6a(size=6.6 K), total size for store is 337.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:54:21,880 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:54:21,880 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1699592061411; duration=0sec 2023-11-10 04:54:21,880 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:21,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5b62c248bc934408a7dfbbf65e02f0c2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/03e32a99f0bb43ffa60a52ba08e12470, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/2cbae9d3d4d94e25b0279478ef43690f, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5bb8cbb7b1ca40889ff0f8d1dd2c5dac, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/98bfc4536bba48dd99cdaf6ed717f09b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/80cbb1f50a0549769178ef759135fbe6, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/6c1795ecbf7d4cb9b77cf06714fd2eb0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/27e4339fdc0f4fd2affbeadef164f79a, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/bcaffe0d3b364ebdbfa8c8539dd97eef, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/d468b5c4a9b540d9a371c223454cd953_SeqId_44_] to archive 2023-11-10 04:54:21,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:21,904 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5b62c248bc934408a7dfbbf65e02f0c2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5b62c248bc934408a7dfbbf65e02f0c2 2023-11-10 04:54:21,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/03e32a99f0bb43ffa60a52ba08e12470 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/03e32a99f0bb43ffa60a52ba08e12470 2023-11-10 04:54:21,907 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/2cbae9d3d4d94e25b0279478ef43690f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/2cbae9d3d4d94e25b0279478ef43690f 2023-11-10 04:54:21,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5bb8cbb7b1ca40889ff0f8d1dd2c5dac to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/5bb8cbb7b1ca40889ff0f8d1dd2c5dac 2023-11-10 04:54:21,909 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/98bfc4536bba48dd99cdaf6ed717f09b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/98bfc4536bba48dd99cdaf6ed717f09b 2023-11-10 04:54:21,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/80cbb1f50a0549769178ef759135fbe6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/80cbb1f50a0549769178ef759135fbe6 2023-11-10 04:54:21,912 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/6c1795ecbf7d4cb9b77cf06714fd2eb0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/6c1795ecbf7d4cb9b77cf06714fd2eb0 2023-11-10 04:54:21,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/27e4339fdc0f4fd2affbeadef164f79a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/27e4339fdc0f4fd2affbeadef164f79a 2023-11-10 04:54:21,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/bcaffe0d3b364ebdbfa8c8539dd97eef to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/bcaffe0d3b364ebdbfa8c8539dd97eef 2023-11-10 04:54:21,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/d468b5c4a9b540d9a371c223454cd953_SeqId_44_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/family1/d468b5c4a9b540d9a371c223454cd953_SeqId_44_ 2023-11-10 04:54:21,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-10 04:54:22,065 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-11-10 04:54:22,066 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 39a447f61d308d5a04d02b6b0b63c0b3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:54:22,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-10 04:54:22,471 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:22,472 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 39a447f61d308d5a04d02b6b0b63c0b3, disabling compactions & flushes 2023-11-10 04:54:22,472 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:22,472 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:22,472 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. after waiting 0 ms 2023-11-10 04:54:22,472 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:22,472 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:22,472 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:22,474 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:54:22,475 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592062475"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592062475"}]},"ts":"1699592062475"} 2023-11-10 04:54:22,476 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-10 04:54:22,477 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:54:22,477 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592062477"}]},"ts":"1699592062477"} 2023-11-10 04:54:22,479 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-11-10 04:54:22,483 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, ASSIGN}] 2023-11-10 04:54:22,485 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, ASSIGN 2023-11-10 04:54:22,486 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:54:22,637 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:22,637 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592062636"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592062636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592062636"}]},"ts":"1699592062636"} 2023-11-10 04:54:22,639 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:22,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-10 04:54:22,790 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:22,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:22,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 39a447f61d308d5a04d02b6b0b63c0b3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:54:22,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:22,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:22,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:22,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:22,795 INFO [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:22,796 DEBUG [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:22,796 DEBUG [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:22,797 INFO [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39a447f61d308d5a04d02b6b0b63c0b3 columnFamilyName family1 2023-11-10 04:54:22,798 INFO [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] regionserver.HStore(310): Store=39a447f61d308d5a04d02b6b0b63c0b3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:22,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:22,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:22,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:22,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:54:22,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 39a447f61d308d5a04d02b6b0b63c0b3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72181519, jitterRate=0.07558844983577728}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:54:22,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:22,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3., pid=41, masterSystemTime=1699592062790 2023-11-10 04:54:22,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:22,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:22,806 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:22,807 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592062806"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592062806"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592062806"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592062806"}]},"ts":"1699592062806"} 2023-11-10 04:54:22,809 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=41, resume processing ppid=40 2023-11-10 04:54:22,809 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485 in 169 msec 2023-11-10 04:54:22,811 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=40, resume processing ppid=39 2023-11-10 04:54:22,811 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, ASSIGN in 326 msec 2023-11-10 04:54:22,812 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:54:22,812 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592062812"}]},"ts":"1699592062812"} 2023-11-10 04:54:22,813 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-11-10 04:54:22,815 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:54:22,816 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1630 sec 2023-11-10 04:54:23,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-10 04:54:23,760 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-11-10 04:54:23,777 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:23,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:23,779 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:23,779 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:23,779 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:23,779 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:23,779 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:23,780 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:23,781 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:23,781 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:23,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:23,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:23,781 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:23,781 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:23,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:23,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:23,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:23,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:23,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:23,786 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:23,786 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:23,786 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:23,786 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:23,786 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:23,786 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:23,786 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:23,786 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:23,787 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-10 04:54:24,207 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511300e44449942d14a539b6c83eded3164d2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511300e44449942d14a539b6c83eded3164d2 2023-11-10 04:54:24,611 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/25d7c0d2e17c473eb8ba52624cc8bdf9 2023-11-10 04:54:24,617 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/25d7c0d2e17c473eb8ba52624cc8bdf9 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/25d7c0d2e17c473eb8ba52624cc8bdf9 2023-11-10 04:54:24,621 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/25d7c0d2e17c473eb8ba52624cc8bdf9, entries=1000, sequenceid=5, filesize=191.4 K 2023-11-10 04:54:24,622 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 835ms, sequenceid=5, compaction requested=false 2023-11-10 04:54:24,622 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:24,623 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:24,623 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:24,623 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:24,623 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:24,623 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,623 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:24,623 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:24,625 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,625 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:24,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:24,625 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,625 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:24,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:24,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:24,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:24,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:24,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d41ba70[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:24,627 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:24,627 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,629 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,629 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:24,629 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:24,630 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:24,630 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,630 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:24,631 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,631 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,631 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:24,631 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:24,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:24,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:24,633 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:24,633 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:24,633 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:24,633 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,633 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:24,635 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,635 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:24,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,635 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:24,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:24,635 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:24,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,636 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:24,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,639 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:24,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:24,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-10 04:54:24,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:24,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:24,639 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:24,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:24,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:24,739 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:24,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:24,744 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:24,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:24,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,746 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:24,746 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:24,746 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:24,746 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:24,747 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,747 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,749 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:24,749 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:24,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:24,749 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:24,749 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:24,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,750 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:24,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,750 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,750 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-10 04:54:24,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:24,751 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:24,751 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:24,751 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-10 04:54:25,164 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511284b70699e9bcc4714a9eccdddcfeca391 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511284b70699e9bcc4714a9eccdddcfeca391 2023-11-10 04:54:25,569 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/9e0c2821d894441ab364d9f57edb65f0 2023-11-10 04:54:25,574 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/9e0c2821d894441ab364d9f57edb65f0 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e0c2821d894441ab364d9f57edb65f0 2023-11-10 04:54:25,579 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e0c2821d894441ab364d9f57edb65f0, entries=1, sequenceid=9, filesize=5.0 K 2023-11-10 04:54:25,580 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 829ms, sequenceid=9, compaction requested=false 2023-11-10 04:54:25,580 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:25,580 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:25,580 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:25,580 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:25,580 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,580 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:25,580 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:25,582 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,582 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:25,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,583 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:25,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:25,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:25,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:25,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:25,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@202a9d8a[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:25,584 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:25,584 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,585 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,585 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:25,585 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:25,586 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:25,586 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,586 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:25,592 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,592 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:25,592 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:25,592 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:25,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:25,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:25,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:25,594 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:25,594 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:25,594 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:25,594 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,594 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:25,596 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,596 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:25,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,596 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:25,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:25,596 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,598 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:25,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:25,603 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:25,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:25,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:25,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:25,603 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:25,603 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:25,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:25,704 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:25,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:25,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:25,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:25,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,710 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:25,710 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:25,710 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:25,710 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:25,710 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,711 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,713 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:25,713 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:25,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:25,713 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:25,713 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:25,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:25,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,714 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-10 04:54:25,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:25,714 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:25,714 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:25,714 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:25,714 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:25,714 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:25,714 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:25,714 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:25,715 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:26,126 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201b033b84dec8749898ec460e4ad856b7a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201b033b84dec8749898ec460e4ad856b7a 2023-11-10 04:54:26,532 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/b6ac2342d4634714b19478b762781ad9 2023-11-10 04:54:26,537 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/b6ac2342d4634714b19478b762781ad9 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/b6ac2342d4634714b19478b762781ad9 2023-11-10 04:54:26,541 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/b6ac2342d4634714b19478b762781ad9, entries=1, sequenceid=13, filesize=5.0 K 2023-11-10 04:54:26,542 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 827ms, sequenceid=13, compaction requested=true 2023-11-10 04:54:26,542 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:26,542 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:26,542 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:26,543 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,544 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:26,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:26,544 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:26,544 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:26,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:26,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:26,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:26,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@251014d7[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:26,545 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:26,545 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,547 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,548 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:26,548 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:26,548 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:26,548 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,548 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:26,549 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:26,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:26,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:26,549 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:26,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:26,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:26,552 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:26,552 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:26,552 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:26,552 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,552 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:26,554 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,554 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:26,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:26,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,554 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:26,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:26,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:26,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:26,554 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,555 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:26,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,558 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,558 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,558 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,558 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,558 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,558 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,558 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,558 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:26,558 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:26,558 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:26,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:26,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:26,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:26,558 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:26,559 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:26,559 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:26,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:26,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:26,659 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:26,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:26,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:26,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:26,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,665 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:26,665 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:26,666 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:26,666 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:26,666 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,666 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,668 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,668 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:26,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:26,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:26,668 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:26,669 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:26,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:26,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,669 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,670 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-10 04:54:26,670 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:26,670 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:26,670 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:26,670 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:26,670 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:26,670 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:26,670 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:26,670 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:26,670 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:27,084 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fd82dc2bc97f46e7b4856f55c44017e7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205fd82dc2bc97f46e7b4856f55c44017e7 2023-11-10 04:54:27,489 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/9e9dd4b651fc4888b13c1ffddbb5c6f9 2023-11-10 04:54:27,494 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/9e9dd4b651fc4888b13c1ffddbb5c6f9 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e9dd4b651fc4888b13c1ffddbb5c6f9 2023-11-10 04:54:27,498 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e9dd4b651fc4888b13c1ffddbb5c6f9, entries=1, sequenceid=17, filesize=5.0 K 2023-11-10 04:54:27,498 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 828ms, sequenceid=17, compaction requested=true 2023-11-10 04:54:27,499 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:27,499 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:27,499 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:27,499 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:27,499 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,499 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:27,499 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:27,501 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,501 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:27,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:27,501 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,501 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:27,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:27,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:27,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:27,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:27,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44afa523[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:27,503 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:27,503 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,504 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,504 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:27,505 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:27,505 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:27,505 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,505 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:27,506 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,506 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:27,506 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:27,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,506 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:27,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:27,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:27,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:27,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:27,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:27,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:27,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:27,509 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:27,509 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:27,509 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:27,509 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,509 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:27,514 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,514 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,514 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:27,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:27,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:27,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:27,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:27,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:27,514 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:27,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,515 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:27,518 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:27,518 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-10 04:54:27,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:27,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:27,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:27,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:27,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:27,518 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:27,619 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:27,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:27,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:27,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:27,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,625 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:27,625 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:27,625 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:27,625 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:27,626 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,626 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,628 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,628 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:27,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:27,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:27,628 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:27,628 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:27,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:27,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,629 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-10 04:54:27,629 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:27,629 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:27,629 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:27,629 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:27,629 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:27,629 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:27,629 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:27,629 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:27,630 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:28,041 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512289d4a199713624a0daff952f90b543bc6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512289d4a199713624a0daff952f90b543bc6 2023-11-10 04:54:28,326 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:54:28,447 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/636c1319d3db4bd9a7b7962bc4c8a62c 2023-11-10 04:54:28,453 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/636c1319d3db4bd9a7b7962bc4c8a62c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/636c1319d3db4bd9a7b7962bc4c8a62c 2023-11-10 04:54:28,458 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/636c1319d3db4bd9a7b7962bc4c8a62c, entries=1, sequenceid=21, filesize=5.0 K 2023-11-10 04:54:28,458 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 829ms, sequenceid=21, compaction requested=true 2023-11-10 04:54:28,459 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:28,459 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:28,459 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:28,459 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:28,459 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,459 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:28,459 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:28,462 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:28,462 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,462 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:28,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:28,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:28,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:28,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:28,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@443d15fc[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:28,464 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:28,464 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,465 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,465 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:28,465 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:28,465 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:28,465 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,465 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:28,467 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:28,467 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,467 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:28,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:28,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:28,467 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:28,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:28,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:28,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:28,469 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:28,469 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:28,469 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:28,469 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,469 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:28,471 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,471 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:28,471 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:28,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:28,471 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:28,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,473 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:28,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,476 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,476 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,476 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,476 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,476 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,476 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,476 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:28,476 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:28,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-10 04:54:28,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:28,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:28,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:28,476 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:28,476 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,477 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:28,477 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:28,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:28,577 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:28,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:28,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:28,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:28,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,583 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:28,583 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:28,583 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:28,583 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:28,583 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,583 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,584 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,584 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:28,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:28,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:28,585 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:28,585 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:28,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,585 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:28,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,586 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,586 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-10 04:54:28,586 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:28,586 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:28,586 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:28,586 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:28,586 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:28,586 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:28,586 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:28,586 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:28,586 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:28,999 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123134832a77958d4921a67ed80c50eadb1e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123134832a77958d4921a67ed80c50eadb1e 2023-11-10 04:54:29,405 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/953dfee37470467788370eada62910b2 2023-11-10 04:54:29,409 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/953dfee37470467788370eada62910b2 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/953dfee37470467788370eada62910b2 2023-11-10 04:54:29,412 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/953dfee37470467788370eada62910b2, entries=1, sequenceid=25, filesize=5.0 K 2023-11-10 04:54:29,413 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 827ms, sequenceid=25, compaction requested=true 2023-11-10 04:54:29,413 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:29,413 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:29,413 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:29,413 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:29,414 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,414 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:29,414 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:29,415 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:29,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:29,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,416 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:29,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:29,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:29,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:29,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:29,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@300f063c[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:29,417 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:29,417 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,419 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,419 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:29,419 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:29,419 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:29,419 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,419 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:29,421 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:29,421 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:29,421 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,421 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:29,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:29,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:29,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:29,423 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:29,423 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:29,423 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:29,423 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:29,423 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,423 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:29,425 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,425 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:29,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:29,425 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:29,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:29,425 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:29,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:29,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:29,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,427 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:29,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:29,429 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:29,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:29,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:29,429 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:29,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:29,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:29,530 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:29,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:29,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:29,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:29,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,535 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:29,535 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:29,535 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:29,535 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:29,536 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,536 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,537 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,537 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:29,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:29,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:29,537 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:29,537 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:29,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:29,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,538 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-10 04:54:29,538 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:29,538 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:29,539 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:29,539 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:29,539 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:29,539 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:29,539 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:29,539 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:29,539 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:29,952 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601010132a04888864a5f8c7551734ad0cf16 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601010132a04888864a5f8c7551734ad0cf16 2023-11-10 04:54:30,357 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/0a2b1200430f434eaaa8c9be352c19b4 2023-11-10 04:54:30,362 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/0a2b1200430f434eaaa8c9be352c19b4 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/0a2b1200430f434eaaa8c9be352c19b4 2023-11-10 04:54:30,366 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/0a2b1200430f434eaaa8c9be352c19b4, entries=1, sequenceid=29, filesize=5.0 K 2023-11-10 04:54:30,367 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 828ms, sequenceid=29, compaction requested=true 2023-11-10 04:54:30,367 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:30,367 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:30,367 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:30,373 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,373 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:30,373 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,373 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:30,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@159c1a46[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:30,375 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:30,375 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,376 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,376 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:30,376 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:30,376 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:30,377 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,377 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:30,381 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:30,381 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,381 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:30,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:30,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:30,381 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:30,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:30,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:30,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:30,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:30,384 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:30,384 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:30,384 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:30,384 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:30,384 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,384 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:30,387 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,387 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:30,387 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:30,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:30,387 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:30,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:30,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,389 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:30,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,391 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,391 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,391 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:30,391 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:30,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:30,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:30,391 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:30,391 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:30,391 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:30,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,392 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:30,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,392 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:30,392 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:30,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:30,492 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:30,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:30,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:30,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:30,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,498 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:30,498 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:30,498 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:30,498 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:30,499 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,499 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,500 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,500 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:30,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:30,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:30,500 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:30,500 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:30,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:30,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,501 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,501 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-10 04:54:30,501 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:30,501 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:30,501 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:30,501 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:30,502 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:30,502 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:30,502 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:30,502 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:30,914 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103f6e7d215812940719d90ff0eeb5f4916 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103f6e7d215812940719d90ff0eeb5f4916 2023-11-10 04:54:31,319 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/2babf1131252483daa8d10a01ad8a453 2023-11-10 04:54:31,324 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/2babf1131252483daa8d10a01ad8a453 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2babf1131252483daa8d10a01ad8a453 2023-11-10 04:54:31,327 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2babf1131252483daa8d10a01ad8a453, entries=1, sequenceid=33, filesize=5.0 K 2023-11-10 04:54:31,328 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 826ms, sequenceid=33, compaction requested=true 2023-11-10 04:54:31,328 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:31,328 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:31,328 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:31,329 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:31,329 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,329 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:31,329 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:31,330 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,331 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:31,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:31,331 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,331 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:31,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:31,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:31,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:31,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:31,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ca3236b[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:31,332 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:31,332 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,335 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:31,335 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:31,335 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:31,335 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,335 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:31,336 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,336 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:31,336 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:31,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:31,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:31,336 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:31,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:31,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:31,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:31,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:31,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:31,339 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:31,339 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:31,339 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:31,339 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,339 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:31,340 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,340 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:31,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,340 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:31,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:31,341 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:31,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,342 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:31,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,345 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,345 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,345 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,345 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:31,345 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:31,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-10 04:54:31,345 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:31,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:31,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:31,346 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:31,345 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:31,345 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,346 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,346 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:31,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:31,346 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:31,346 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:31,446 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:31,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:31,451 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:31,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:31,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,452 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:31,452 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:31,452 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:31,452 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:31,453 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,453 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:31,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:31,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:31,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:31,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:31,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:31,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,456 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,456 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-10 04:54:31,456 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:31,456 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:31,456 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:31,456 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:31,457 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:31,457 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:31,457 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:31,457 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:31,457 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:31,869 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090729391ad1fcc94ba88c84228925f7a9d1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015090729391ad1fcc94ba88c84228925f7a9d1 2023-11-10 04:54:32,274 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/dd726f96b5384c97b50542038d732f25 2023-11-10 04:54:32,279 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/dd726f96b5384c97b50542038d732f25 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/dd726f96b5384c97b50542038d732f25 2023-11-10 04:54:32,283 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/dd726f96b5384c97b50542038d732f25, entries=1, sequenceid=37, filesize=5.0 K 2023-11-10 04:54:32,284 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 827ms, sequenceid=37, compaction requested=true 2023-11-10 04:54:32,284 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:32,284 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:32,284 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:32,284 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:32,284 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,284 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:32,284 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:32,286 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,286 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:32,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:32,286 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,286 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:32,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:32,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:32,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:32,288 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:32,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d51f6[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:32,288 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,289 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:32,289 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:32,289 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,289 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:32,291 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:32,291 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:32,291 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,291 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:32,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:32,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:32,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:32,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:32,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:32,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:32,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:32,294 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:32,294 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:32,294 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:32,294 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,294 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:32,297 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,297 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,297 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:32,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:32,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:32,297 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:32,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,299 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:32,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:32,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-10 04:54:32,304 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:32,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:32,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:32,304 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:32,305 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:32,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:32,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:32,405 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:32,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:32,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:32,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:32,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,411 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:32,411 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:32,411 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-10 04:54:32,411 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-10 04:54:32,412 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,412 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,413 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:32,413 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:32,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:32,414 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:32,414 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:32,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,415 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-10 04:54:32,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:32,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-10 04:54:32,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:32,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-10 04:54:32,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:32,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:32,415 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:32,415 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. started... 2023-11-10 04:54:32,415 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 39a447f61d308d5a04d02b6b0b63c0b3 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:32,827 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ac8baf696b06487f90be4706a8b4a0ef to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120ac8baf696b06487f90be4706a8b4a0ef 2023-11-10 04:54:33,233 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/d35c8cf8233f4db19435b7851979f024 2023-11-10 04:54:33,237 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/d35c8cf8233f4db19435b7851979f024 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/d35c8cf8233f4db19435b7851979f024 2023-11-10 04:54:33,241 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/d35c8cf8233f4db19435b7851979f024, entries=1, sequenceid=41, filesize=5.0 K 2023-11-10 04:54:33,242 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 39a447f61d308d5a04d02b6b0b63c0b3 in 827ms, sequenceid=41, compaction requested=true 2023-11-10 04:54:33,242 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:33,242 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:33,242 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:33,242 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:33,242 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,242 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-10 04:54:33,242 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-10 04:54:33,245 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:33,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:33,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:33,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-10 04:54:33,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@486b992f[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:33,246 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-10 04:54:33,247 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,248 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,248 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-10 04:54:33,248 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-10 04:54:33,248 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:33,248 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,248 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:33,249 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,249 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-10 04:54:33,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,249 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:33,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:33,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:33,249 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-10 04:54:33,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:33,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:33,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:33,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:33,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-10 04:54:33,252 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-10 04:54:33,252 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:33,252 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:33,252 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,252 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:33,253 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,253 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:33,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:33,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,253 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:33,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:33,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:33,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:33,254 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:33,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,255 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:33,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,258 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,258 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:33,258 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,258 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:33,258 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:33,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:33,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:33,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:33,258 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:33,258 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:33,258 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,258 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:33,259 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:33,258 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,259 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:33,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:33,359 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:33,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:33,361 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:33,362 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-10 04:54:33,363 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-10 04:54:33,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:33,363 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592673363 2023-11-10 04:54:33,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:54:33,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-10 04:54:33,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:33,368 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x5b58dd49 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:33,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7b7b5440, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:33,372 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:34,208 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c0c634e50ed54f6a8aa4234e7867c3dc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103c0c634e50ed54f6a8aa4234e7867c3dc 2023-11-10 04:54:34,209 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:34,221 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:54:34,222 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50556, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:54:34,228 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:54:34,230 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34746, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:54:34,244 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bffa8aaa7fc340e8806407c070c33ade first=Optional[r3] last=Optional[r6] 2023-11-10 04:54:34,244 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bffa8aaa7fc340e8806407c070c33ade}] 2023-11-10 04:54:34,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bffa8aaa7fc340e8806407c070c33ade for inclusion in 39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:34,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-10 04:54:34,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:34,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:34,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bffa8aaa7fc340e8806407c070c33ade to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rrvs2hve2dvms2e5gph8mr2otkb57t3bs3os5l92reenb0ggans3bos1ld42i8ui/family1/bffa8aaa7fc340e8806407c070c33ade 2023-11-10 04:54:34,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rrvs2hve2dvms2e5gph8mr2otkb57t3bs3os5l92reenb0ggans3bos1ld42i8ui/family1/bffa8aaa7fc340e8806407c070c33ade as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9ffd46163d4c4c3882e154e764bf3e5b_SeqId_44_ 2023-11-10 04:54:34,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rrvs2hve2dvms2e5gph8mr2otkb57t3bs3os5l92reenb0ggans3bos1ld42i8ui/family1/bffa8aaa7fc340e8806407c070c33ade into 39a447f61d308d5a04d02b6b0b63c0b3/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9ffd46163d4c4c3882e154e764bf3e5b_SeqId_44_ - updating store file list. 2023-11-10 04:54:34,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9ffd46163d4c4c3882e154e764bf3e5b_SeqId_44_ into 39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:34,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rrvs2hve2dvms2e5gph8mr2otkb57t3bs3os5l92reenb0ggans3bos1ld42i8ui/family1/bffa8aaa7fc340e8806407c070c33ade into 39a447f61d308d5a04d02b6b0b63c0b3/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9ffd46163d4c4c3882e154e764bf3e5b_SeqId_44_) 2023-11-10 04:54:34,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rrvs2hve2dvms2e5gph8mr2otkb57t3bs3os5l92reenb0ggans3bos1ld42i8ui/family1/bffa8aaa7fc340e8806407c070c33ade 2023-11-10 04:54:34,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-10 04:54:34,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-10 04:54:34,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 39a447f61d308d5a04d02b6b0b63c0b3/family1 is initiating minor compaction 2023-11-10 04:54:34,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2af469a6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:34,262 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 39a447f61d308d5a04d02b6b0b63c0b3/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:34,262 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e0c2821d894441ab364d9f57edb65f0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/b6ac2342d4634714b19478b762781ad9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e9dd4b651fc4888b13c1ffddbb5c6f9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/636c1319d3db4bd9a7b7962bc4c8a62c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/953dfee37470467788370eada62910b2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/0a2b1200430f434eaaa8c9be352c19b4, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2babf1131252483daa8d10a01ad8a453, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/dd726f96b5384c97b50542038d732f25, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/d35c8cf8233f4db19435b7851979f024, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9ffd46163d4c4c3882e154e764bf3e5b_SeqId_44_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp, totalSize=50.4 K 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 9e0c2821d894441ab364d9f57edb65f0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting b6ac2342d4634714b19478b762781ad9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 9e9dd4b651fc4888b13c1ffddbb5c6f9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 636c1319d3db4bd9a7b7962bc4c8a62c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 953dfee37470467788370eada62910b2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 0a2b1200430f434eaaa8c9be352c19b4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 2babf1131252483daa8d10a01ad8a453, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting dd726f96b5384c97b50542038d732f25, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting d35c8cf8233f4db19435b7851979f024, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-10 04:54:34,262 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 9ffd46163d4c4c3882e154e764bf3e5b_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-10 04:54:34,264 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:34,264 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:34,266 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512289d4a199713624a0daff952f90b543bc6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512289d4a199713624a0daff952f90b543bc6 2023-11-10 04:54:34,268 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123134832a77958d4921a67ed80c50eadb1e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123134832a77958d4921a67ed80c50eadb1e 2023-11-10 04:54:34,269 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601010132a04888864a5f8c7551734ad0cf16 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601010132a04888864a5f8c7551734ad0cf16 2023-11-10 04:54:34,270 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103f6e7d215812940719d90ff0eeb5f4916 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103f6e7d215812940719d90ff0eeb5f4916 2023-11-10 04:54:34,270 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-11-10 04:54:34,270 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-10 04:54:34,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:34,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:34,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:34,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:34,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:34,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:34,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:34,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:34,294 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 39a447f61d308d5a04d02b6b0b63c0b3#family1#compaction#60 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:54:35,112 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/2f1c5b1401d54d049b252646a590c147 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2f1c5b1401d54d049b252646a590c147 2023-11-10 04:54:35,118 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 39a447f61d308d5a04d02b6b0b63c0b3/family1 of 39a447f61d308d5a04d02b6b0b63c0b3 into 2f1c5b1401d54d049b252646a590c147(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:54:35,118 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:35,118 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1699592074262; duration=0sec 2023-11-10 04:54:35,118 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:35,127 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120527c9633036de4d2382647378926fa302 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120527c9633036de4d2382647378926fa302 2023-11-10 04:54:35,127 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:35,149 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0bee0bd73bcd433d808ad709efb45bad first=Optional[r00] last=Optional[r2] 2023-11-10 04:54:35,150 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0bee0bd73bcd433d808ad709efb45bad}] 2023-11-10 04:54:35,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0bee0bd73bcd433d808ad709efb45bad for inclusion in 39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:35,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-10 04:54:35,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:35,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:35,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0bee0bd73bcd433d808ad709efb45bad to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rej9sl21n9vki27r59e1qjpith7jnr96pdptt1c637l4ai5efjmfisur1016r21q/family1/0bee0bd73bcd433d808ad709efb45bad 2023-11-10 04:54:35,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rej9sl21n9vki27r59e1qjpith7jnr96pdptt1c637l4ai5efjmfisur1016r21q/family1/0bee0bd73bcd433d808ad709efb45bad as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2ddbba8c237149a2b2a44be0f12d9a93_SeqId_47_ 2023-11-10 04:54:35,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rej9sl21n9vki27r59e1qjpith7jnr96pdptt1c637l4ai5efjmfisur1016r21q/family1/0bee0bd73bcd433d808ad709efb45bad into 39a447f61d308d5a04d02b6b0b63c0b3/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2ddbba8c237149a2b2a44be0f12d9a93_SeqId_47_ - updating store file list. 2023-11-10 04:54:35,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2ddbba8c237149a2b2a44be0f12d9a93_SeqId_47_ into 39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:35,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rej9sl21n9vki27r59e1qjpith7jnr96pdptt1c637l4ai5efjmfisur1016r21q/family1/0bee0bd73bcd433d808ad709efb45bad into 39a447f61d308d5a04d02b6b0b63c0b3/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2ddbba8c237149a2b2a44be0f12d9a93_SeqId_47_) 2023-11-10 04:54:35,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__rej9sl21n9vki27r59e1qjpith7jnr96pdptt1c637l4ai5efjmfisur1016r21q/family1/0bee0bd73bcd433d808ad709efb45bad 2023-11-10 04:54:35,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-10 04:54:35,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-10 04:54:35,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 39a447f61d308d5a04d02b6b0b63c0b3/family1 is initiating minor compaction (all files) 2023-11-10 04:54:35,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@64d5aea8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:35,162 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 39a447f61d308d5a04d02b6b0b63c0b3/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:35,162 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/25d7c0d2e17c473eb8ba52624cc8bdf9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2f1c5b1401d54d049b252646a590c147, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2ddbba8c237149a2b2a44be0f12d9a93_SeqId_47_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp, totalSize=390.1 K 2023-11-10 04:54:35,162 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 25d7c0d2e17c473eb8ba52624cc8bdf9, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878827283 2023-11-10 04:54:35,163 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 2f1c5b1401d54d049b252646a590c147, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621227283 2023-11-10 04:54:35,163 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 2ddbba8c237149a2b2a44be0f12d9a93_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878827283 2023-11-10 04:54:35,164 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:35,164 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:35,166 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511300e44449942d14a539b6c83eded3164d2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511300e44449942d14a539b6c83eded3164d2 2023-11-10 04:54:35,167 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201b033b84dec8749898ec460e4ad856b7a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201b033b84dec8749898ec460e4ad856b7a 2023-11-10 04:54:35,168 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205fd82dc2bc97f46e7b4856f55c44017e7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205fd82dc2bc97f46e7b4856f55c44017e7 2023-11-10 04:54:35,168 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-11-10 04:54:35,168 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-10 04:54:35,169 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:35,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:35,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:35,170 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-10 04:54:35,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8090 sec 2023-11-10 04:54:35,178 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 39a447f61d308d5a04d02b6b0b63c0b3#family1#compaction#61 average throughput is 25.80 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:54:35,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e0c2821d894441ab364d9f57edb65f0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/b6ac2342d4634714b19478b762781ad9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e9dd4b651fc4888b13c1ffddbb5c6f9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/636c1319d3db4bd9a7b7962bc4c8a62c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/953dfee37470467788370eada62910b2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/0a2b1200430f434eaaa8c9be352c19b4, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2babf1131252483daa8d10a01ad8a453, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/dd726f96b5384c97b50542038d732f25, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/d35c8cf8233f4db19435b7851979f024, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9ffd46163d4c4c3882e154e764bf3e5b_SeqId_44_] to archive 2023-11-10 04:54:35,212 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:35,214 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e0c2821d894441ab364d9f57edb65f0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e0c2821d894441ab364d9f57edb65f0 2023-11-10 04:54:35,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/b6ac2342d4634714b19478b762781ad9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/b6ac2342d4634714b19478b762781ad9 2023-11-10 04:54:35,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e9dd4b651fc4888b13c1ffddbb5c6f9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9e9dd4b651fc4888b13c1ffddbb5c6f9 2023-11-10 04:54:35,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/636c1319d3db4bd9a7b7962bc4c8a62c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/636c1319d3db4bd9a7b7962bc4c8a62c 2023-11-10 04:54:35,218 DEBUG [hfile_cleaner-dir-scan-pool-1] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 41 more 2023-11-10 04:54:35,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/953dfee37470467788370eada62910b2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/953dfee37470467788370eada62910b2 2023-11-10 04:54:35,220 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/0a2b1200430f434eaaa8c9be352c19b4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/0a2b1200430f434eaaa8c9be352c19b4 2023-11-10 04:54:35,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2babf1131252483daa8d10a01ad8a453 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2babf1131252483daa8d10a01ad8a453 2023-11-10 04:54:35,223 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/dd726f96b5384c97b50542038d732f25 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/dd726f96b5384c97b50542038d732f25 2023-11-10 04:54:35,224 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/d35c8cf8233f4db19435b7851979f024 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/d35c8cf8233f4db19435b7851979f024 2023-11-10 04:54:35,225 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9ffd46163d4c4c3882e154e764bf3e5b_SeqId_44_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/9ffd46163d4c4c3882e154e764bf3e5b_SeqId_44_ 2023-11-10 04:54:35,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-10 04:54:35,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:35,670 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-10 04:54:35,671 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-11-10 04:54:35,671 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-10 04:54:35,673 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-11-10 04:54:35,676 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, REOPEN/MOVE}] 2023-11-10 04:54:35,676 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, REOPEN/MOVE 2023-11-10 04:54:35,677 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:35,677 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592075677"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592075677"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592075677"}]},"ts":"1699592075677"} 2023-11-10 04:54:35,678 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:35,829 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:35,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:35,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 39a447f61d308d5a04d02b6b0b63c0b3, disabling compactions & flushes 2023-11-10 04:54:35,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:36,396 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/.tmp/family1/1756d3a8353e4ed1b02b5652c3d605f5 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/1756d3a8353e4ed1b02b5652c3d605f5 2023-11-10 04:54:36,400 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 39a447f61d308d5a04d02b6b0b63c0b3/family1 of 39a447f61d308d5a04d02b6b0b63c0b3 into 1756d3a8353e4ed1b02b5652c3d605f5(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:54:36,400 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:36,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:36,401 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1699592075162; duration=1sec 2023-11-10 04:54:36,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:36,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. after waiting 0 ms 2023-11-10 04:54:36,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:36,401 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:36,401 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/25d7c0d2e17c473eb8ba52624cc8bdf9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2f1c5b1401d54d049b252646a590c147, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2ddbba8c237149a2b2a44be0f12d9a93_SeqId_47_] to archive 2023-11-10 04:54:36,402 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:36,403 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/25d7c0d2e17c473eb8ba52624cc8bdf9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/25d7c0d2e17c473eb8ba52624cc8bdf9 2023-11-10 04:54:36,404 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2f1c5b1401d54d049b252646a590c147 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2f1c5b1401d54d049b252646a590c147 2023-11-10 04:54:36,405 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2ddbba8c237149a2b2a44be0f12d9a93_SeqId_47_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/2ddbba8c237149a2b2a44be0f12d9a93_SeqId_47_ 2023-11-10 04:54:36,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-10 04:54:36,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:36,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:36,411 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 39a447f61d308d5a04d02b6b0b63c0b3 to self. 2023-11-10 04:54:36,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:36,413 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=CLOSED 2023-11-10 04:54:36,413 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592076413"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592076413"}]},"ts":"1699592076413"} 2023-11-10 04:54:36,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=46, resume processing ppid=45 2023-11-10 04:54:36,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485 in 736 msec 2023-11-10 04:54:36,416 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=true 2023-11-10 04:54:36,566 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:36,566 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592076566"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592076566"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592076566"}]},"ts":"1699592076566"} 2023-11-10 04:54:36,568 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:36,719 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:36,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:36,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 39a447f61d308d5a04d02b6b0b63c0b3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:54:36,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:36,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:36,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:36,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:36,724 INFO [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:36,725 DEBUG [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:36,725 DEBUG [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:36,725 INFO [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39a447f61d308d5a04d02b6b0b63c0b3 columnFamilyName family1 2023-11-10 04:54:36,730 DEBUG [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/1756d3a8353e4ed1b02b5652c3d605f5 2023-11-10 04:54:36,730 INFO [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] regionserver.HStore(310): Store=39a447f61d308d5a04d02b6b0b63c0b3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:36,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:36,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:36,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:36,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 39a447f61d308d5a04d02b6b0b63c0b3; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65532285, jitterRate=-0.023492857813835144}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:54:36,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:36,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3., pid=47, masterSystemTime=1699592076719 2023-11-10 04:54:36,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:36,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:36,737 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:36,737 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592076737"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592076737"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592076737"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592076737"}]},"ts":"1699592076737"} 2023-11-10 04:54:36,739 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=47, resume processing ppid=45 2023-11-10 04:54:36,739 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485 in 170 msec 2023-11-10 04:54:36,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=45, resume processing ppid=44 2023-11-10 04:54:36,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, REOPEN/MOVE in 1.0630 sec 2023-11-10 04:54:36,743 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=44, resume processing ppid=43 2023-11-10 04:54:36,743 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0680 sec 2023-11-10 04:54:36,744 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.4830 sec 2023-11-10 04:54:36,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=43 2023-11-10 04:54:36,746 INFO [Listener at localhost/35765] compactions.TestMobCompactor(278): alter status finished 2023-11-10 04:54:36,747 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:36,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-10 04:54:36,749 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-10 04:54:36,749 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:36,749 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592676749 2023-11-10 04:54:36,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:54:36,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-10 04:54:36,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:36,752 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x60200550 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:36,756 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@41d89440, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:36,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:36,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:36,757 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:36,757 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:36,757 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:37,585 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ef2e31650eea4edaac37b951a777cb70 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128ef2e31650eea4edaac37b951a777cb70 2023-11-10 04:54:37,587 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:37,599 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:54:37,600 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57302, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:54:37,606 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:54:37,607 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47586, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:54:37,616 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/36f668da59cc4b18a1a9aea1d446446c first=Optional[r05] last=Optional[r8] 2023-11-10 04:54:37,616 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=51 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/36f668da59cc4b18a1a9aea1d446446c}] 2023-11-10 04:54:37,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/36f668da59cc4b18a1a9aea1d446446c for inclusion in 39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:37,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-10 04:54:37,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:37,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:37,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/36f668da59cc4b18a1a9aea1d446446c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kvg6g6e4v25apl0c30t1antd9a6nh7rrpq9trvod99qel8reonvr0sr2tdsd25rp/family1/36f668da59cc4b18a1a9aea1d446446c 2023-11-10 04:54:37,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kvg6g6e4v25apl0c30t1antd9a6nh7rrpq9trvod99qel8reonvr0sr2tdsd25rp/family1/36f668da59cc4b18a1a9aea1d446446c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/f1eecac976924fc4842f43ea010ac05e_SeqId_53_ 2023-11-10 04:54:37,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kvg6g6e4v25apl0c30t1antd9a6nh7rrpq9trvod99qel8reonvr0sr2tdsd25rp/family1/36f668da59cc4b18a1a9aea1d446446c into 39a447f61d308d5a04d02b6b0b63c0b3/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/f1eecac976924fc4842f43ea010ac05e_SeqId_53_ - updating store file list. 2023-11-10 04:54:37,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/f1eecac976924fc4842f43ea010ac05e_SeqId_53_ into 39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:37,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kvg6g6e4v25apl0c30t1antd9a6nh7rrpq9trvod99qel8reonvr0sr2tdsd25rp/family1/36f668da59cc4b18a1a9aea1d446446c into 39a447f61d308d5a04d02b6b0b63c0b3/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/f1eecac976924fc4842f43ea010ac05e_SeqId_53_) 2023-11-10 04:54:37,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kvg6g6e4v25apl0c30t1antd9a6nh7rrpq9trvod99qel8reonvr0sr2tdsd25rp/family1/36f668da59cc4b18a1a9aea1d446446c 2023-11-10 04:54:37,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:37,633 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:37,635 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120ac8baf696b06487f90be4706a8b4a0ef to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120ac8baf696b06487f90be4706a8b4a0ef 2023-11-10 04:54:37,636 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511284b70699e9bcc4714a9eccdddcfeca391 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511284b70699e9bcc4714a9eccdddcfeca391 2023-11-10 04:54:37,636 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-11-10 04:54:37,636 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-10 04:54:37,637 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:37,637 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:37,637 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:37,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:37,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:37,638 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-10 04:54:37,639 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 891 msec 2023-11-10 04:54:37,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-10 04:54:37,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:38,098 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-11-10 04:54:38,099 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-10 04:54:38,099 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-11-10 04:54:38,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-11-10 04:54:38,103 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, REOPEN/MOVE}] 2023-11-10 04:54:38,104 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, REOPEN/MOVE 2023-11-10 04:54:38,105 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:38,105 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592078105"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592078105"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592078105"}]},"ts":"1699592078105"} 2023-11-10 04:54:38,106 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:38,258 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:38,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:38,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 39a447f61d308d5a04d02b6b0b63c0b3, disabling compactions & flushes 2023-11-10 04:54:38,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:38,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:38,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. after waiting 0 ms 2023-11-10 04:54:38,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:38,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-11-10 04:54:38,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:38,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:38,264 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 39a447f61d308d5a04d02b6b0b63c0b3 to self. 2023-11-10 04:54:38,266 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:38,266 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=CLOSED 2023-11-10 04:54:38,266 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592078266"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592078266"}]},"ts":"1699592078266"} 2023-11-10 04:54:38,268 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=52, resume processing ppid=51 2023-11-10 04:54:38,269 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485 in 161 msec 2023-11-10 04:54:38,269 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=true 2023-11-10 04:54:38,419 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:38,420 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592078419"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592078419"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592078419"}]},"ts":"1699592078419"} 2023-11-10 04:54:38,421 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:38,573 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:38,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:38,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 39a447f61d308d5a04d02b6b0b63c0b3, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:54:38,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:38,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:38,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:38,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:38,577 INFO [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:38,578 DEBUG [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:38,578 DEBUG [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1 2023-11-10 04:54:38,578 INFO [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39a447f61d308d5a04d02b6b0b63c0b3 columnFamilyName family1 2023-11-10 04:54:38,584 DEBUG [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/1756d3a8353e4ed1b02b5652c3d605f5 2023-11-10 04:54:38,587 DEBUG [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/family1/f1eecac976924fc4842f43ea010ac05e_SeqId_53_ 2023-11-10 04:54:38,587 INFO [StoreOpener-39a447f61d308d5a04d02b6b0b63c0b3-1] regionserver.HStore(310): Store=39a447f61d308d5a04d02b6b0b63c0b3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:38,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:38,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:38,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:54:38,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 39a447f61d308d5a04d02b6b0b63c0b3; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70884953, jitterRate=0.056268110871315}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:54:38,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:54:38,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3., pid=53, masterSystemTime=1699592078573 2023-11-10 04:54:38,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:38,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:54:38,594 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=39a447f61d308d5a04d02b6b0b63c0b3, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:38,594 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1699592078594"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592078594"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592078594"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592078594"}]},"ts":"1699592078594"} 2023-11-10 04:54:38,596 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=53, resume processing ppid=51 2023-11-10 04:54:38,597 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 39a447f61d308d5a04d02b6b0b63c0b3, server=jenkins-hbase4.apache.org,36419,1699592002485 in 174 msec 2023-11-10 04:54:38,598 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=51, resume processing ppid=50 2023-11-10 04:54:38,598 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=39a447f61d308d5a04d02b6b0b63c0b3, REOPEN/MOVE in 494 msec 2023-11-10 04:54:38,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=50, resume processing ppid=49 2023-11-10 04:54:38,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 497 msec 2023-11-10 04:54:38,601 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 912 msec 2023-11-10 04:54:38,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-11-10 04:54:38,603 INFO [Listener at localhost/35765] compactions.TestMobCompactor(278): alter status finished 2023-11-10 04:54:38,604 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-10 04:54:38,604 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-10 04:54:38,605 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-10 04:54:38,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:38,606 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592678606 2023-11-10 04:54:38,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:54:38,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-10 04:54:38,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:38,608 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x4aba4dc9 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:38,612 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4fc5ba4a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:38,612 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:38,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:38,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:38,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:38,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:38,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:38,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:38,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-10 04:54:38,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-10 04:54:38,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:38,614 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-10 04:54:38,615 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 10 msec 2023-11-10 04:54:38,662 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=251 (was 257), OpenFileDescriptor=660 (was 656) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=142 (was 183), ProcessCount=167 (was 170), AvailableMemoryMB=10826 (was 10831) 2023-11-10 04:54:38,672 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=251, OpenFileDescriptor=660, MaxFileDescriptor=60000, SystemLoadAverage=142, ProcessCount=167, AvailableMemoryMB=10825 2023-11-10 04:54:38,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-10 04:54:38,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:38,676 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:54:38,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-11-10 04:54:38,677 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:54:38,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-10 04:54:38,678 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:38,678 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f empty. 2023-11-10 04:54:38,679 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:38,679 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-11-10 04:54:38,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-10 04:54:38,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-10 04:54:39,086 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-11-10 04:54:39,087 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2b8f43a41c5d5687d6c3e84194ded34f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:54:39,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-10 04:54:39,493 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:39,493 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 2b8f43a41c5d5687d6c3e84194ded34f, disabling compactions & flushes 2023-11-10 04:54:39,493 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:39,493 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:39,493 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. after waiting 0 ms 2023-11-10 04:54:39,493 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:39,493 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:39,493 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:39,495 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:54:39,496 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592079496"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592079496"}]},"ts":"1699592079496"} 2023-11-10 04:54:39,497 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-10 04:54:39,498 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:54:39,498 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592079498"}]},"ts":"1699592079498"} 2023-11-10 04:54:39,499 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-11-10 04:54:39,503 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, ASSIGN}] 2023-11-10 04:54:39,504 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, ASSIGN 2023-11-10 04:54:39,505 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:54:39,656 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:39,656 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592079656"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592079656"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592079656"}]},"ts":"1699592079656"} 2023-11-10 04:54:39,657 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:39,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-10 04:54:39,809 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:39,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:39,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2b8f43a41c5d5687d6c3e84194ded34f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:54:39,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:39,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:39,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:39,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:39,813 INFO [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:39,814 DEBUG [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:39,814 DEBUG [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:39,815 INFO [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2b8f43a41c5d5687d6c3e84194ded34f columnFamilyName family1 2023-11-10 04:54:39,816 INFO [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] regionserver.HStore(310): Store=2b8f43a41c5d5687d6c3e84194ded34f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:39,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:39,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:39,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:39,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:54:39,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2b8f43a41c5d5687d6c3e84194ded34f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66166263, jitterRate=-0.014045849442481995}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:54:39,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:39,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f., pid=57, masterSystemTime=1699592079808 2023-11-10 04:54:39,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:39,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:39,824 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:39,824 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592079824"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592079824"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592079824"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592079824"}]},"ts":"1699592079824"} 2023-11-10 04:54:39,827 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=57, resume processing ppid=56 2023-11-10 04:54:39,827 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485 in 168 msec 2023-11-10 04:54:39,828 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=56, resume processing ppid=55 2023-11-10 04:54:39,828 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, ASSIGN in 324 msec 2023-11-10 04:54:39,829 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:54:39,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592079829"}]},"ts":"1699592079829"} 2023-11-10 04:54:39,830 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-11-10 04:54:39,833 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:54:39,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1590 sec 2023-11-10 04:54:40,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-10 04:54:40,782 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-11-10 04:54:40,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:40,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:40,804 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:40,804 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:40,804 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:40,804 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:40,805 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:40,805 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:40,806 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:40,806 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:40,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:40,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:40,807 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:40,807 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:40,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:40,807 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:40,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:40,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:40,808 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:40,808 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:40,808 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:40,808 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:40,808 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:40,810 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:40,810 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:40,810 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:40,810 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:40,810 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-10 04:54:41,229 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113050de3d6f7468437dae7b58c3d6bb2003 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113050de3d6f7468437dae7b58c3d6bb2003 2023-11-10 04:54:41,633 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/7ca233152b974156a4c53df84a5317a9 2023-11-10 04:54:41,638 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/7ca233152b974156a4c53df84a5317a9 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7ca233152b974156a4c53df84a5317a9 2023-11-10 04:54:41,642 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7ca233152b974156a4c53df84a5317a9, entries=1000, sequenceid=5, filesize=190.5 K 2023-11-10 04:54:41,643 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 833ms, sequenceid=5, compaction requested=false 2023-11-10 04:54:41,643 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:41,644 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:41,644 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:41,644 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:41,645 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:41,645 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,645 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:41,645 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:41,646 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,646 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:41,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:41,647 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,647 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:41,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:41,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:41,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:41,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:41,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dd0e44e[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:41,649 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:41,649 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,650 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,650 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:41,650 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:41,650 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:41,650 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,650 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:41,652 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:41,652 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,652 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:41,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:41,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:41,652 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:41,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:41,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:41,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:41,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:41,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:41,655 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:41,655 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:41,655 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:41,655 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,655 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:41,661 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,661 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:41,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,661 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:41,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:41,661 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:41,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,663 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:41,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,665 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,665 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,665 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:41,665 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:41,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:41,665 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:41,665 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-10 04:54:41,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:41,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:41,665 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:41,665 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:41,666 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:41,665 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,666 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:41,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:41,666 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,666 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,766 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:41,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:41,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:41,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:41,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,772 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:41,772 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:41,772 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:41,772 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:41,772 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,773 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,775 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,775 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:41,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:41,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:41,775 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:41,775 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:41,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:41,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-10 04:54:41,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:41,776 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:41,776 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:41,776 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:41,777 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:41,777 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:41,777 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:41,777 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:41,777 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-10 04:54:41,941 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:54:42,189 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511288b119f5d667f490f95fc90c814b5e6f9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511288b119f5d667f490f95fc90c814b5e6f9 2023-11-10 04:54:42,594 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/7270e9a317ef429faa3107d38d4a9d35 2023-11-10 04:54:42,599 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/7270e9a317ef429faa3107d38d4a9d35 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7270e9a317ef429faa3107d38d4a9d35 2023-11-10 04:54:42,603 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7270e9a317ef429faa3107d38d4a9d35, entries=1, sequenceid=9, filesize=5.0 K 2023-11-10 04:54:42,604 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 827ms, sequenceid=9, compaction requested=false 2023-11-10 04:54:42,604 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:42,604 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:42,604 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:42,604 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:42,604 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,604 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:42,604 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:42,606 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,606 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:42,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:42,606 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,606 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:42,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:42,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:42,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:42,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:42,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8dd20e4[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:42,608 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:42,608 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,609 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,609 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:42,609 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:42,609 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:42,610 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,610 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:42,611 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:42,611 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:42,611 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,611 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:42,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:42,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:42,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:42,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:42,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:42,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:42,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:42,614 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:42,614 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:42,614 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:42,614 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,614 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:42,617 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,617 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:42,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:42,617 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:42,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:42,617 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:42,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:42,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:42,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,619 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:42,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,621 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,621 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,621 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,621 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,621 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,621 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,621 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,621 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:42,621 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:42,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:42,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-10 04:54:42,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:42,621 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:42,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:42,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:42,622 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:42,622 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:42,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:42,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:42,673 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-10 04:54:42,722 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:42,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:42,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:42,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:42,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,727 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:42,727 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:42,727 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:42,727 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:42,728 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,728 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,729 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:42,729 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:42,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:42,730 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:42,730 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:42,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:42,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,730 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,731 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-10 04:54:42,731 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:42,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:42,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:42,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:42,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:42,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:42,731 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:42,731 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:42,731 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:43,144 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512019f57bda749bb44c79e6c955d5440416f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512019f57bda749bb44c79e6c955d5440416f 2023-11-10 04:54:43,550 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/6d2a051e78664d428823efc27381b22e 2023-11-10 04:54:43,554 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/6d2a051e78664d428823efc27381b22e as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/6d2a051e78664d428823efc27381b22e 2023-11-10 04:54:43,558 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/6d2a051e78664d428823efc27381b22e, entries=1, sequenceid=13, filesize=5.0 K 2023-11-10 04:54:43,559 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 828ms, sequenceid=13, compaction requested=true 2023-11-10 04:54:43,559 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:43,559 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:43,561 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,561 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:43,561 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:43,562 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:43,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:43,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@622542e8[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:43,563 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:43,563 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,564 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,564 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:43,564 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:43,564 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:43,564 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,564 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:43,566 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,566 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:43,566 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:43,566 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:43,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:43,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:43,568 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:43,568 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:43,568 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:43,568 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,568 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:43,571 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,571 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,571 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:43,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:43,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:43,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:43,571 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:43,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,573 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:43,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,577 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,577 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:43,577 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:43,577 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:43,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:43,577 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:43,577 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,577 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:43,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:43,577 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,578 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,578 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,578 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:43,578 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:43,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:43,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:43,678 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:43,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:43,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:43,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:43,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,684 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:43,684 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:43,684 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:43,684 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:43,684 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,684 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,688 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:43,688 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:43,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:43,689 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:43,689 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,689 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-10 04:54:43,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:43,690 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:43,690 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:43,690 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:43,690 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:44,103 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120504633a9338bd4cff8f2d40da2e15ddf6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120504633a9338bd4cff8f2d40da2e15ddf6 2023-11-10 04:54:44,108 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/3c2535d3c22d44d1abe91dda5172a6a0 2023-11-10 04:54:44,113 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/3c2535d3c22d44d1abe91dda5172a6a0 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c2535d3c22d44d1abe91dda5172a6a0 2023-11-10 04:54:44,116 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c2535d3c22d44d1abe91dda5172a6a0, entries=1, sequenceid=17, filesize=5.0 K 2023-11-10 04:54:44,117 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 427ms, sequenceid=17, compaction requested=true 2023-11-10 04:54:44,117 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:44,117 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:44,120 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,120 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:44,120 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,120 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:44,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:44,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:44,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76131d62[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:44,122 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:44,122 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:44,123 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:44,124 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,124 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:44,125 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:44,125 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,125 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:44,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:44,125 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:44,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:44,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:44,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:44,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:44,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:44,128 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:44,128 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:44,128 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:44,128 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,128 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:44,130 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,130 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:44,130 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:44,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:44,131 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:44,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,132 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:44,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,135 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,135 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:44,135 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:44,135 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:44,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:44,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:44,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:44,136 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:44,136 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,136 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,136 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,136 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,136 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:44,136 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:44,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:44,136 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:44,236 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:44,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:44,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:44,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:44,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,242 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:44,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:44,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:44,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:44,243 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,243 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,244 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:44,244 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:44,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:44,244 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:44,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,244 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:44,245 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:44,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,245 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,245 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-10 04:54:44,245 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:44,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:44,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:44,245 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:44,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:44,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:44,246 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:44,246 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:44,246 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:44,657 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512280a7cbb4a7f234cacaa0f51da4223635d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512280a7cbb4a7f234cacaa0f51da4223635d 2023-11-10 04:54:45,062 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/407e1120dbf84463913706cd68e5c87c 2023-11-10 04:54:45,067 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/407e1120dbf84463913706cd68e5c87c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/407e1120dbf84463913706cd68e5c87c 2023-11-10 04:54:45,071 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/407e1120dbf84463913706cd68e5c87c, entries=1, sequenceid=21, filesize=5.0 K 2023-11-10 04:54:45,071 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 825ms, sequenceid=21, compaction requested=true 2023-11-10 04:54:45,071 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:45,072 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:45,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:45,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:45,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:45,072 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:45,073 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,073 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:45,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:45,074 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,074 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:45,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:45,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:45,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:45,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:45,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c5b757a[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:45,075 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:45,075 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,077 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,077 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:45,077 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:45,077 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:45,077 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,077 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:45,080 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,080 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:45,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:45,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:45,080 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:45,080 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:45,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:45,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:45,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:45,082 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:45,082 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:45,082 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:45,082 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,082 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:45,084 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,084 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,084 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:45,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:45,084 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:45,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,086 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:45,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:45,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:45,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:45,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:45,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:45,088 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:45,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,088 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:45,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:45,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:45,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,088 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:45,089 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,089 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,089 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,189 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:45,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:45,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:45,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:45,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,194 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:45,195 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:45,195 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:45,195 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:45,195 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,195 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,196 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:45,196 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:45,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:45,197 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:45,197 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:45,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:45,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-10 04:54:45,198 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:45,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:45,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:45,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:45,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:45,198 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:45,198 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:45,198 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:45,198 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:45,609 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231329380a7a6964e9e8b2033feb6453ed1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231329380a7a6964e9e8b2033feb6453ed1 2023-11-10 04:54:46,015 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/2955522b647243e1857e8570ea74eb8a 2023-11-10 04:54:46,019 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/2955522b647243e1857e8570ea74eb8a as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/2955522b647243e1857e8570ea74eb8a 2023-11-10 04:54:46,023 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/2955522b647243e1857e8570ea74eb8a, entries=1, sequenceid=25, filesize=5.0 K 2023-11-10 04:54:46,023 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 825ms, sequenceid=25, compaction requested=true 2023-11-10 04:54:46,024 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:46,024 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:46,024 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:46,024 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:46,024 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,024 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:46,024 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:46,027 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,027 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:46,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:46,027 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,027 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:46,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:46,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:46,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:46,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:46,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2de0b615[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:46,028 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:46,028 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,029 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,030 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:46,030 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:46,030 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:46,030 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,030 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:46,031 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:46,031 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,031 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:46,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:46,031 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:46,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:46,034 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:46,034 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:46,034 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:46,034 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,034 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:46,035 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,035 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,035 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:46,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:46,036 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:46,037 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:46,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:46,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,040 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:46,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:46,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:46,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:46,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:46,040 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-10 04:54:46,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:46,041 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:46,041 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:46,141 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:46,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:46,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:46,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:46,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,147 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:46,147 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:46,147 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:46,147 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:46,148 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,148 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,149 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:46,149 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:46,149 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,149 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:46,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,150 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,150 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-10 04:54:46,150 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,151 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:46,151 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:46,151 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:46,151 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:46,151 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:46,151 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:46,151 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:46,151 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:46,563 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101d498aa3378424e0cb0fa4ec80534d616 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101d498aa3378424e0cb0fa4ec80534d616 2023-11-10 04:54:46,968 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/4c0004ac679a4250808473418e06a609 2023-11-10 04:54:46,973 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/4c0004ac679a4250808473418e06a609 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/4c0004ac679a4250808473418e06a609 2023-11-10 04:54:46,977 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/4c0004ac679a4250808473418e06a609, entries=1, sequenceid=29, filesize=5.0 K 2023-11-10 04:54:46,978 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 827ms, sequenceid=29, compaction requested=true 2023-11-10 04:54:46,978 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:46,978 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:46,978 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:46,978 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:46,978 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,978 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:46,978 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:46,980 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,980 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:46,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:46,980 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,980 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:46,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:46,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:46,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:46,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:46,982 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@651a9343[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:46,982 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:46,982 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,983 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,983 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:46,983 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:46,983 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:46,983 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,983 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:46,986 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:46,986 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,986 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:46,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:46,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:46,986 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:46,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:46,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:46,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:46,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:46,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:46,988 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:46,988 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:46,988 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:46,988 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,988 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:46,990 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,990 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:46,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:46,990 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:46,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:46,991 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:46,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:46,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:46,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,992 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:46,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,994 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,994 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,994 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:46,994 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,994 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:46,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-10 04:54:46,994 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:46,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:46,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:46,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:46,994 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:46,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:46,994 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,995 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,995 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:46,995 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:46,995 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:46,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:46,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:47,095 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:47,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:47,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:47,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:47,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,101 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:47,101 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:47,101 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:47,101 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:47,102 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,102 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,103 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:47,103 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:47,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:47,104 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,104 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:47,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:47,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,104 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,105 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-10 04:54:47,105 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:47,105 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:47,117 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:47,117 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:47,117 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:47,117 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:47,117 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:47,534 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010342d79e35216249d88bf7bc064d5879d0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010342d79e35216249d88bf7bc064d5879d0 2023-11-10 04:54:47,939 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/3c880e9e6a444c638e49296e0715de95 2023-11-10 04:54:47,944 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/3c880e9e6a444c638e49296e0715de95 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c880e9e6a444c638e49296e0715de95 2023-11-10 04:54:47,948 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c880e9e6a444c638e49296e0715de95, entries=1, sequenceid=33, filesize=5.0 K 2023-11-10 04:54:47,949 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 832ms, sequenceid=33, compaction requested=true 2023-11-10 04:54:47,949 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:47,949 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:47,952 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,952 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:47,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:47,952 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,952 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:47,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:47,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:47,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:47,954 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:47,954 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b0c6a8f[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:47,954 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:47,954 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,955 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,955 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:47,955 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:47,955 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:47,955 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,955 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:47,959 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:47,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:47,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:47,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:47,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:47,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:47,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:47,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:47,962 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:47,962 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:47,962 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:47,962 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,962 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:47,963 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,963 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:47,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:47,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,963 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:47,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:47,964 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:47,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:47,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,965 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:47,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:47,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:47,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:47,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:47,968 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:47,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:47,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:47,968 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:48,068 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:48,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:48,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:48,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:48,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,075 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:48,075 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:48,075 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:48,075 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:48,076 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,076 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,078 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:48,078 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:48,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:48,078 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,078 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:48,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:48,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,079 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,079 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-10 04:54:48,079 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:48,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:48,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:48,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:48,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:48,079 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:48,080 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:48,080 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:48,495 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075f2c0ec9e04346368c07ff7a01fb8fef to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509075f2c0ec9e04346368c07ff7a01fb8fef 2023-11-10 04:54:48,901 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/fca077daa7614fd88df4ddea9020301c 2023-11-10 04:54:48,906 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/fca077daa7614fd88df4ddea9020301c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/fca077daa7614fd88df4ddea9020301c 2023-11-10 04:54:48,909 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/fca077daa7614fd88df4ddea9020301c, entries=1, sequenceid=37, filesize=5.0 K 2023-11-10 04:54:48,910 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 830ms, sequenceid=37, compaction requested=true 2023-11-10 04:54:48,910 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:48,910 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:48,910 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:48,910 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:48,910 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,910 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:48,910 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:48,912 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,912 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:48,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:48,912 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,912 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:48,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:48,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:48,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:48,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:48,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ceb2e31[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:48,914 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:48,914 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,915 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:48,915 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:48,915 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:48,915 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,915 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:48,916 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:48,917 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,917 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:48,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:48,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:48,917 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:48,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:48,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:48,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:48,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:48,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:48,919 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:48,919 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:48,919 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:48,919 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,919 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:48,922 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,922 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:48,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:48,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,922 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:48,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:48,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:48,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:48,922 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:48,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,923 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:48,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,926 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,926 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:48,926 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,926 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:48,926 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:48,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:48,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:48,926 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:48,927 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:48,927 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,927 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:48,927 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,927 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:48,927 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:48,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:49,027 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:49,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:49,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:54:49,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-10 04:54:49,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,033 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:49,033 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:49,033 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-10 04:54:49,033 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-10 04:54:49,034 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,034 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,035 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:49,035 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:49,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:49,035 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,036 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:54:49,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:54:49,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,037 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-10 04:54:49,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-10 04:54:49,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:54:49,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-10 04:54:49,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:54:49,037 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:54:49,037 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:49,037 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. started... 2023-11-10 04:54:49,037 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b8f43a41c5d5687d6c3e84194ded34f 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:54:49,449 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511204995df20428e42baab87c7e09c5f0070 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511204995df20428e42baab87c7e09c5f0070 2023-11-10 04:54:49,854 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/7c6eec7724b34f21a65531716882273d 2023-11-10 04:54:49,859 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/7c6eec7724b34f21a65531716882273d as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7c6eec7724b34f21a65531716882273d 2023-11-10 04:54:49,864 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7c6eec7724b34f21a65531716882273d, entries=1, sequenceid=41, filesize=5.0 K 2023-11-10 04:54:49,865 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2b8f43a41c5d5687d6c3e84194ded34f in 828ms, sequenceid=41, compaction requested=true 2023-11-10 04:54:49,865 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:49,865 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:54:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:54:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-10 04:54:49,865 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-10 04:54:49,867 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,867 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:49,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:49,867 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,867 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:54:49,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:49,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:49,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:49,869 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-10 04:54:49,869 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@341373a7[Count = 0] remaining members to acquire global barrier 2023-11-10 04:54:49,869 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-10 04:54:49,869 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,872 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,872 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-10 04:54:49,872 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-10 04:54:49,872 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:54:49,873 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,873 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:54:49,874 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-10 04:54:49,874 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:49,874 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,874 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-10 04:54:49,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:49,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:49,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:49,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:49,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:49,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:54:49,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-10 04:54:49,877 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-10 04:54:49,877 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:54:49,877 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:54:49,877 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,877 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:54:49,879 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:54:49,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:54:49,879 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,879 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:49,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:54:49,879 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:49,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:49,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:54:49,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,881 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:54:49,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,886 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,886 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:54:49,886 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:54:49,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:54:49,886 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:54:49,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:54:49,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:54:49,886 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:54:49,886 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,886 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:49,886 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,886 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-10 04:54:49,887 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:54:49,886 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,887 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:54:49,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:54:49,987 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:54:49,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:54:49,988 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:49,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-10 04:54:49,991 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-10 04:54:49,991 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:49,991 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592689991 2023-11-10 04:54:49,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:54:49,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-10 04:54:49,996 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:49,997 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x5fdb7e68 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:50,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@38ff9ea9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:50,002 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:50,852 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103449896c374cb469cb94ea9ca1c223bce to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103449896c374cb469cb94ea9ca1c223bce 2023-11-10 04:54:50,853 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:50,865 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:54:50,866 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46944, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:54:50,873 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:54:50,874 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51008, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:54:50,881 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d2c0a14a2d3e4fa79ab040e6c490f7cc first=Optional[r3] last=Optional[r6] 2023-11-10 04:54:50,882 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d2c0a14a2d3e4fa79ab040e6c490f7cc}] 2023-11-10 04:54:50,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d2c0a14a2d3e4fa79ab040e6c490f7cc for inclusion in 2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:50,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-10 04:54:50,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:50,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:50,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d2c0a14a2d3e4fa79ab040e6c490f7cc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__95bg1iffd9s1mut345el60r5k3vs3qpo8a5l78ntuhqfus3e27sje6oosnecipqt/family1/d2c0a14a2d3e4fa79ab040e6c490f7cc 2023-11-10 04:54:50,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__95bg1iffd9s1mut345el60r5k3vs3qpo8a5l78ntuhqfus3e27sje6oosnecipqt/family1/d2c0a14a2d3e4fa79ab040e6c490f7cc as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/8e7555f0f5dc45f68ad7a8bb206ca438_SeqId_44_ 2023-11-10 04:54:50,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__95bg1iffd9s1mut345el60r5k3vs3qpo8a5l78ntuhqfus3e27sje6oosnecipqt/family1/d2c0a14a2d3e4fa79ab040e6c490f7cc into 2b8f43a41c5d5687d6c3e84194ded34f/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/8e7555f0f5dc45f68ad7a8bb206ca438_SeqId_44_ - updating store file list. 2023-11-10 04:54:50,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/8e7555f0f5dc45f68ad7a8bb206ca438_SeqId_44_ into 2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:50,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__95bg1iffd9s1mut345el60r5k3vs3qpo8a5l78ntuhqfus3e27sje6oosnecipqt/family1/d2c0a14a2d3e4fa79ab040e6c490f7cc into 2b8f43a41c5d5687d6c3e84194ded34f/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/8e7555f0f5dc45f68ad7a8bb206ca438_SeqId_44_) 2023-11-10 04:54:50,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__95bg1iffd9s1mut345el60r5k3vs3qpo8a5l78ntuhqfus3e27sje6oosnecipqt/family1/d2c0a14a2d3e4fa79ab040e6c490f7cc 2023-11-10 04:54:50,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-10 04:54:50,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-10 04:54:50,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 2b8f43a41c5d5687d6c3e84194ded34f/family1 is initiating minor compaction 2023-11-10 04:54:50,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7128d012; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:50,897 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2b8f43a41c5d5687d6c3e84194ded34f/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:50,897 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7270e9a317ef429faa3107d38d4a9d35, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/6d2a051e78664d428823efc27381b22e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c2535d3c22d44d1abe91dda5172a6a0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/407e1120dbf84463913706cd68e5c87c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/2955522b647243e1857e8570ea74eb8a, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/4c0004ac679a4250808473418e06a609, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c880e9e6a444c638e49296e0715de95, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/fca077daa7614fd88df4ddea9020301c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7c6eec7724b34f21a65531716882273d, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/8e7555f0f5dc45f68ad7a8bb206ca438_SeqId_44_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp, totalSize=50.3 K 2023-11-10 04:54:50,897 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 7270e9a317ef429faa3107d38d4a9d35, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-10 04:54:50,898 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 6d2a051e78664d428823efc27381b22e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-10 04:54:50,898 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 3c2535d3c22d44d1abe91dda5172a6a0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-10 04:54:50,898 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 407e1120dbf84463913706cd68e5c87c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-10 04:54:50,898 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 2955522b647243e1857e8570ea74eb8a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-10 04:54:50,898 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 4c0004ac679a4250808473418e06a609, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-10 04:54:50,898 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 3c880e9e6a444c638e49296e0715de95, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-10 04:54:50,898 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting fca077daa7614fd88df4ddea9020301c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-10 04:54:50,898 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 7c6eec7724b34f21a65531716882273d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-10 04:54:50,898 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 8e7555f0f5dc45f68ad7a8bb206ca438_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-10 04:54:50,899 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:50,900 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:50,902 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512280a7cbb4a7f234cacaa0f51da4223635d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512280a7cbb4a7f234cacaa0f51da4223635d 2023-11-10 04:54:50,903 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231329380a7a6964e9e8b2033feb6453ed1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231329380a7a6964e9e8b2033feb6453ed1 2023-11-10 04:54:50,904 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101d498aa3378424e0cb0fa4ec80534d616 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101d498aa3378424e0cb0fa4ec80534d616 2023-11-10 04:54:50,905 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010342d79e35216249d88bf7bc064d5879d0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010342d79e35216249d88bf7bc064d5879d0 2023-11-10 04:54:50,905 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-11-10 04:54:50,905 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-10 04:54:50,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:50,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:50,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:50,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:50,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:50,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:50,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:50,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:50,929 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2b8f43a41c5d5687d6c3e84194ded34f#family1#compaction#72 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:54:50,951 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4aba4dc9 to 127.0.0.1:51169 2023-11-10 04:54:50,952 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:50,952 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:50,952 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x60200550 to 127.0.0.1:51169 2023-11-10 04:54:50,952 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:50,952 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:50,953 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5b58dd49 to 127.0.0.1:51169 2023-11-10 04:54:50,953 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:50,953 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:50,953 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1142a68c to 127.0.0.1:51169 2023-11-10 04:54:50,953 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:50,953 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:50,953 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x43607900 to 127.0.0.1:51169 2023-11-10 04:54:50,953 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:51,382 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/1a9f698076714836840385ed64d3caab as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/1a9f698076714836840385ed64d3caab 2023-11-10 04:54:51,387 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2b8f43a41c5d5687d6c3e84194ded34f/family1 of 2b8f43a41c5d5687d6c3e84194ded34f into 1a9f698076714836840385ed64d3caab(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:54:51,387 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:51,387 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1699592090897; duration=0sec 2023-11-10 04:54:51,387 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:51,402 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7270e9a317ef429faa3107d38d4a9d35, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/6d2a051e78664d428823efc27381b22e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c2535d3c22d44d1abe91dda5172a6a0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/407e1120dbf84463913706cd68e5c87c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/2955522b647243e1857e8570ea74eb8a, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/4c0004ac679a4250808473418e06a609, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c880e9e6a444c638e49296e0715de95, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/fca077daa7614fd88df4ddea9020301c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7c6eec7724b34f21a65531716882273d, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/8e7555f0f5dc45f68ad7a8bb206ca438_SeqId_44_] to archive 2023-11-10 04:54:51,403 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:51,404 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7270e9a317ef429faa3107d38d4a9d35 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7270e9a317ef429faa3107d38d4a9d35 2023-11-10 04:54:51,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/6d2a051e78664d428823efc27381b22e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/6d2a051e78664d428823efc27381b22e 2023-11-10 04:54:51,406 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c2535d3c22d44d1abe91dda5172a6a0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c2535d3c22d44d1abe91dda5172a6a0 2023-11-10 04:54:51,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/407e1120dbf84463913706cd68e5c87c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/407e1120dbf84463913706cd68e5c87c 2023-11-10 04:54:51,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/2955522b647243e1857e8570ea74eb8a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/2955522b647243e1857e8570ea74eb8a 2023-11-10 04:54:51,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/4c0004ac679a4250808473418e06a609 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/4c0004ac679a4250808473418e06a609 2023-11-10 04:54:51,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c880e9e6a444c638e49296e0715de95 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/3c880e9e6a444c638e49296e0715de95 2023-11-10 04:54:51,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/fca077daa7614fd88df4ddea9020301c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/fca077daa7614fd88df4ddea9020301c 2023-11-10 04:54:51,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7c6eec7724b34f21a65531716882273d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7c6eec7724b34f21a65531716882273d 2023-11-10 04:54:51,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/8e7555f0f5dc45f68ad7a8bb206ca438_SeqId_44_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/8e7555f0f5dc45f68ad7a8bb206ca438_SeqId_44_ 2023-11-10 04:54:51,785 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051f08cfffbf4640f7a2d01459b92958a9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512051f08cfffbf4640f7a2d01459b92958a9 2023-11-10 04:54:51,785 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:51,808 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/6f38f096ca4e44d3be8c839fa7194608 first=Optional[r00] last=Optional[r2] 2023-11-10 04:54:51,809 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/6f38f096ca4e44d3be8c839fa7194608}] 2023-11-10 04:54:51,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/6f38f096ca4e44d3be8c839fa7194608 for inclusion in 2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:51,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-10 04:54:51,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:51,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:51,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/6f38f096ca4e44d3be8c839fa7194608 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rebfv6ncq6eeekbmdkc3jlhb2p330s20t7dpbl76cakk0c4jbiqm43kqmmjg4kcd/family1/6f38f096ca4e44d3be8c839fa7194608 2023-11-10 04:54:51,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rebfv6ncq6eeekbmdkc3jlhb2p330s20t7dpbl76cakk0c4jbiqm43kqmmjg4kcd/family1/6f38f096ca4e44d3be8c839fa7194608 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b20a6117fa84c59b8f98d8d20167237_SeqId_47_ 2023-11-10 04:54:51,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rebfv6ncq6eeekbmdkc3jlhb2p330s20t7dpbl76cakk0c4jbiqm43kqmmjg4kcd/family1/6f38f096ca4e44d3be8c839fa7194608 into 2b8f43a41c5d5687d6c3e84194ded34f/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b20a6117fa84c59b8f98d8d20167237_SeqId_47_ - updating store file list. 2023-11-10 04:54:51,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b20a6117fa84c59b8f98d8d20167237_SeqId_47_ into 2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:51,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rebfv6ncq6eeekbmdkc3jlhb2p330s20t7dpbl76cakk0c4jbiqm43kqmmjg4kcd/family1/6f38f096ca4e44d3be8c839fa7194608 into 2b8f43a41c5d5687d6c3e84194ded34f/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b20a6117fa84c59b8f98d8d20167237_SeqId_47_) 2023-11-10 04:54:51,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rebfv6ncq6eeekbmdkc3jlhb2p330s20t7dpbl76cakk0c4jbiqm43kqmmjg4kcd/family1/6f38f096ca4e44d3be8c839fa7194608 2023-11-10 04:54:51,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-10 04:54:51,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-10 04:54:51,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 2b8f43a41c5d5687d6c3e84194ded34f/family1 is initiating minor compaction (all files) 2023-11-10 04:54:51,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4a78574c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:51,825 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2b8f43a41c5d5687d6c3e84194ded34f/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:51,825 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7ca233152b974156a4c53df84a5317a9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/1a9f698076714836840385ed64d3caab, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b20a6117fa84c59b8f98d8d20167237_SeqId_47_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp, totalSize=388.1 K 2023-11-10 04:54:51,825 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 7ca233152b974156a4c53df84a5317a9, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878827283 2023-11-10 04:54:51,826 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 1a9f698076714836840385ed64d3caab, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621227283 2023-11-10 04:54:51,826 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 9b20a6117fa84c59b8f98d8d20167237_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878827283 2023-11-10 04:54:51,827 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:51,827 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:51,829 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113050de3d6f7468437dae7b58c3d6bb2003 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113050de3d6f7468437dae7b58c3d6bb2003 2023-11-10 04:54:51,830 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512019f57bda749bb44c79e6c955d5440416f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512019f57bda749bb44c79e6c955d5440416f 2023-11-10 04:54:51,831 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120504633a9338bd4cff8f2d40da2e15ddf6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120504633a9338bd4cff8f2d40da2e15ddf6 2023-11-10 04:54:51,831 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-11-10 04:54:51,831 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-10 04:54:51,832 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:51,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:51,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:51,833 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-10 04:54:51,835 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8440 sec 2023-11-10 04:54:51,844 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2b8f43a41c5d5687d6c3e84194ded34f#family1#compaction#73 average throughput is 19.96 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:54:51,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-10 04:54:51,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:52,316 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-10 04:54:52,317 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-11-10 04:54:52,317 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-10 04:54:52,319 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-11-10 04:54:52,321 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, REOPEN/MOVE}] 2023-11-10 04:54:52,322 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, REOPEN/MOVE 2023-11-10 04:54:52,323 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:52,323 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592092322"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592092322"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592092322"}]},"ts":"1699592092322"} 2023-11-10 04:54:52,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:52,475 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:52,476 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:52,476 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2b8f43a41c5d5687d6c3e84194ded34f, disabling compactions & flushes 2023-11-10 04:54:52,476 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:52,664 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/.tmp/family1/a7d094e5c8194402ac883f261a1745ec as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/a7d094e5c8194402ac883f261a1745ec 2023-11-10 04:54:52,669 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 2b8f43a41c5d5687d6c3e84194ded34f/family1 of 2b8f43a41c5d5687d6c3e84194ded34f into a7d094e5c8194402ac883f261a1745ec(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:54:52,669 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:52,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:52,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:52,669 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1699592091825; duration=0sec 2023-11-10 04:54:52,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. after waiting 0 ms 2023-11-10 04:54:52,669 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:52,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:52,672 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7ca233152b974156a4c53df84a5317a9, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/1a9f698076714836840385ed64d3caab, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b20a6117fa84c59b8f98d8d20167237_SeqId_47_] to archive 2023-11-10 04:54:52,673 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:52,674 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7ca233152b974156a4c53df84a5317a9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/7ca233152b974156a4c53df84a5317a9 2023-11-10 04:54:52,676 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/1a9f698076714836840385ed64d3caab to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/1a9f698076714836840385ed64d3caab 2023-11-10 04:54:52,677 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b20a6117fa84c59b8f98d8d20167237_SeqId_47_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b20a6117fa84c59b8f98d8d20167237_SeqId_47_ 2023-11-10 04:54:52,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-10 04:54:52,681 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:52,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:52,681 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 2b8f43a41c5d5687d6c3e84194ded34f to self. 2023-11-10 04:54:52,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:52,683 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=CLOSED 2023-11-10 04:54:52,683 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592092683"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592092683"}]},"ts":"1699592092683"} 2023-11-10 04:54:52,685 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=62, resume processing ppid=61 2023-11-10 04:54:52,685 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485 in 360 msec 2023-11-10 04:54:52,685 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=true 2023-11-10 04:54:52,836 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:52,836 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592092836"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592092836"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592092836"}]},"ts":"1699592092836"} 2023-11-10 04:54:52,837 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:52,989 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:52,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:52,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2b8f43a41c5d5687d6c3e84194ded34f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:54:52,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:52,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:52,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:52,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:52,993 INFO [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:52,994 DEBUG [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:52,994 DEBUG [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:52,994 INFO [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2b8f43a41c5d5687d6c3e84194ded34f columnFamilyName family1 2023-11-10 04:54:52,999 DEBUG [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/a7d094e5c8194402ac883f261a1745ec 2023-11-10 04:54:52,999 INFO [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] regionserver.HStore(310): Store=2b8f43a41c5d5687d6c3e84194ded34f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:53,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:53,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:53,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:53,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2b8f43a41c5d5687d6c3e84194ded34f; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65888625, jitterRate=-0.01818297803401947}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:54:53,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:53,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f., pid=63, masterSystemTime=1699592092989 2023-11-10 04:54:53,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:53,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:53,006 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:53,006 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592093006"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592093006"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592093006"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592093006"}]},"ts":"1699592093006"} 2023-11-10 04:54:53,009 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=63, resume processing ppid=61 2023-11-10 04:54:53,009 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485 in 170 msec 2023-11-10 04:54:53,010 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=61, resume processing ppid=60 2023-11-10 04:54:53,010 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, REOPEN/MOVE in 688 msec 2023-11-10 04:54:53,011 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=60, resume processing ppid=59 2023-11-10 04:54:53,011 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 691 msec 2023-11-10 04:54:53,013 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1060 sec 2023-11-10 04:54:53,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=59 2023-11-10 04:54:53,015 INFO [Listener at localhost/35765] compactions.TestMobCompactor(278): alter status finished 2023-11-10 04:54:53,016 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:53,017 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-10 04:54:53,018 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-10 04:54:53,018 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:53,018 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592693018 2023-11-10 04:54:53,019 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:54:53,019 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-10 04:54:53,019 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:53,020 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x714066be to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:53,025 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e79baf6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:53,025 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:53,025 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:53,025 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:53,025 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:53,025 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:53,848 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112891c477c8c5934f0eaa9778088c5607ba to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112891c477c8c5934f0eaa9778088c5607ba 2023-11-10 04:54:53,849 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:53,861 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:54:53,862 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46954, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:54:53,868 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:54:53,870 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51012, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:54:53,877 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c466861b9f73446b9104bfbac925f110 first=Optional[r05] last=Optional[r8] 2023-11-10 04:54:53,878 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=51 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c466861b9f73446b9104bfbac925f110}] 2023-11-10 04:54:53,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c466861b9f73446b9104bfbac925f110 for inclusion in 2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:53,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-10 04:54:53,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:53,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:53,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/c466861b9f73446b9104bfbac925f110 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fl8lsvjd3f3dgsc19qhjhpnpv05ifnv57hk38tg6e8i4e20eekkivll8krsg3ij3/family1/c466861b9f73446b9104bfbac925f110 2023-11-10 04:54:53,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fl8lsvjd3f3dgsc19qhjhpnpv05ifnv57hk38tg6e8i4e20eekkivll8krsg3ij3/family1/c466861b9f73446b9104bfbac925f110 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b2c5c87eb784c9ab88b6c5976018c14_SeqId_53_ 2023-11-10 04:54:53,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fl8lsvjd3f3dgsc19qhjhpnpv05ifnv57hk38tg6e8i4e20eekkivll8krsg3ij3/family1/c466861b9f73446b9104bfbac925f110 into 2b8f43a41c5d5687d6c3e84194ded34f/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b2c5c87eb784c9ab88b6c5976018c14_SeqId_53_ - updating store file list. 2023-11-10 04:54:53,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b2c5c87eb784c9ab88b6c5976018c14_SeqId_53_ into 2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:53,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fl8lsvjd3f3dgsc19qhjhpnpv05ifnv57hk38tg6e8i4e20eekkivll8krsg3ij3/family1/c466861b9f73446b9104bfbac925f110 into 2b8f43a41c5d5687d6c3e84194ded34f/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b2c5c87eb784c9ab88b6c5976018c14_SeqId_53_) 2023-11-10 04:54:53,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fl8lsvjd3f3dgsc19qhjhpnpv05ifnv57hk38tg6e8i4e20eekkivll8krsg3ij3/family1/c466861b9f73446b9104bfbac925f110 2023-11-10 04:54:53,893 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:53,894 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:53,895 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511204995df20428e42baab87c7e09c5f0070 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511204995df20428e42baab87c7e09c5f0070 2023-11-10 04:54:53,896 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511288b119f5d667f490f95fc90c814b5e6f9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511288b119f5d667f490f95fc90c814b5e6f9 2023-11-10 04:54:53,896 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-11-10 04:54:53,896 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-10 04:54:53,897 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:53,897 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:53,897 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:53,897 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:53,897 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:53,898 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-10 04:54:53,899 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 882 msec 2023-11-10 04:54:53,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-11-10 04:54:53,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:54,353 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-11-10 04:54:54,354 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-10 04:54:54,354 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-11-10 04:54:54,356 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-11-10 04:54:54,358 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, REOPEN/MOVE}] 2023-11-10 04:54:54,358 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, REOPEN/MOVE 2023-11-10 04:54:54,359 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:54,359 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592094359"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592094359"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592094359"}]},"ts":"1699592094359"} 2023-11-10 04:54:54,360 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:54,512 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:54,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:54,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2b8f43a41c5d5687d6c3e84194ded34f, disabling compactions & flushes 2023-11-10 04:54:54,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:54,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:54,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. after waiting 0 ms 2023-11-10 04:54:54,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:54,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-11-10 04:54:54,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:54,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:54,518 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 2b8f43a41c5d5687d6c3e84194ded34f to self. 2023-11-10 04:54:54,519 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:54,519 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=CLOSED 2023-11-10 04:54:54,519 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592094519"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592094519"}]},"ts":"1699592094519"} 2023-11-10 04:54:54,521 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=68, resume processing ppid=67 2023-11-10 04:54:54,521 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485 in 160 msec 2023-11-10 04:54:54,521 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=true 2023-11-10 04:54:54,672 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:54,672 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592094672"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592094672"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592094672"}]},"ts":"1699592094672"} 2023-11-10 04:54:54,674 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:54,825 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:54,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:54,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2b8f43a41c5d5687d6c3e84194ded34f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:54:54,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:54,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:54,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:54,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:54,829 INFO [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:54,830 DEBUG [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:54,830 DEBUG [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1 2023-11-10 04:54:54,830 INFO [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2b8f43a41c5d5687d6c3e84194ded34f columnFamilyName family1 2023-11-10 04:54:54,835 DEBUG [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/9b2c5c87eb784c9ab88b6c5976018c14_SeqId_53_ 2023-11-10 04:54:54,837 DEBUG [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/family1/a7d094e5c8194402ac883f261a1745ec 2023-11-10 04:54:54,838 INFO [StoreOpener-2b8f43a41c5d5687d6c3e84194ded34f-1] regionserver.HStore(310): Store=2b8f43a41c5d5687d6c3e84194ded34f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:54,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:54,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:54,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:54:54,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2b8f43a41c5d5687d6c3e84194ded34f; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69941348, jitterRate=0.042207300662994385}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:54:54,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:54:54,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f., pid=69, masterSystemTime=1699592094825 2023-11-10 04:54:54,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:54,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:54:54,844 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=2b8f43a41c5d5687d6c3e84194ded34f, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:54,845 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1699592094844"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592094844"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592094844"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592094844"}]},"ts":"1699592094844"} 2023-11-10 04:54:54,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=69, resume processing ppid=67 2023-11-10 04:54:54,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 2b8f43a41c5d5687d6c3e84194ded34f, server=jenkins-hbase4.apache.org,36419,1699592002485 in 172 msec 2023-11-10 04:54:54,848 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=67, resume processing ppid=66 2023-11-10 04:54:54,848 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=2b8f43a41c5d5687d6c3e84194ded34f, REOPEN/MOVE in 489 msec 2023-11-10 04:54:54,850 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=66, resume processing ppid=65 2023-11-10 04:54:54,850 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 493 msec 2023-11-10 04:54:54,851 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 906 msec 2023-11-10 04:54:54,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=65 2023-11-10 04:54:54,853 INFO [Listener at localhost/35765] compactions.TestMobCompactor(278): alter status finished 2023-11-10 04:54:54,854 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-10 04:54:54,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-10 04:54:54,856 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-10 04:54:54,856 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:54,856 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592694856 2023-11-10 04:54:54,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:54:54,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-10 04:54:54,858 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:54,860 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x2addf24f to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:54,865 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2439eefb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:54,865 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:54,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:54,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:54,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:54,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:54,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:54,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:54,866 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-10 04:54:54,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-10 04:54:54,866 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:54,867 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-10 04:54:54,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 12 msec 2023-11-10 04:54:54,909 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=244 (was 251), OpenFileDescriptor=653 (was 660), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=118 (was 142), ProcessCount=167 (was 167), AvailableMemoryMB=10795 (was 10825) 2023-11-10 04:54:54,920 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=244, OpenFileDescriptor=653, MaxFileDescriptor=60000, SystemLoadAverage=118, ProcessCount=167, AvailableMemoryMB=10794 2023-11-10 04:54:54,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-10 04:54:54,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-11-10 04:54:54,926 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:54:54,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-11-10 04:54:54,926 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:54:54,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-10 04:54:54,928 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:54:54,928 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b empty. 2023-11-10 04:54:54,928 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:54:54,928 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-11-10 04:54:55,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-10 04:54:55,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-10 04:54:55,336 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-11-10 04:54:55,337 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3b0bf194570ba8a438c5807e175dcf6b, NAME => 'testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:54:55,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-10 04:54:55,742 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:55,743 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 3b0bf194570ba8a438c5807e175dcf6b, disabling compactions & flushes 2023-11-10 04:54:55,743 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:54:55,743 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:54:55,743 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. after waiting 0 ms 2023-11-10 04:54:55,743 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:54:55,743 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:54:55,743 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 3b0bf194570ba8a438c5807e175dcf6b: 2023-11-10 04:54:55,745 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:54:55,746 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1699592095746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592095746"}]},"ts":"1699592095746"} 2023-11-10 04:54:55,747 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-10 04:54:55,747 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:54:55,747 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592095747"}]},"ts":"1699592095747"} 2023-11-10 04:54:55,748 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-11-10 04:54:55,751 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=3b0bf194570ba8a438c5807e175dcf6b, ASSIGN}] 2023-11-10 04:54:55,752 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=3b0bf194570ba8a438c5807e175dcf6b, ASSIGN 2023-11-10 04:54:55,753 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=3b0bf194570ba8a438c5807e175dcf6b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:54:55,904 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=3b0bf194570ba8a438c5807e175dcf6b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:55,904 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1699592095904"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592095904"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592095904"}]},"ts":"1699592095904"} 2023-11-10 04:54:55,905 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 3b0bf194570ba8a438c5807e175dcf6b, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:54:56,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-10 04:54:56,056 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:56,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:54:56,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3b0bf194570ba8a438c5807e175dcf6b, NAME => 'testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:54:56,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-11-10 04:54:56,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-11-10 04:54:56,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:54:56,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:54:56,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:54:56,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:54:56,063 INFO [StoreOpener-3b0bf194570ba8a438c5807e175dcf6b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:54:56,064 DEBUG [StoreOpener-3b0bf194570ba8a438c5807e175dcf6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1 2023-11-10 04:54:56,064 DEBUG [StoreOpener-3b0bf194570ba8a438c5807e175dcf6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1 2023-11-10 04:54:56,065 INFO [StoreOpener-3b0bf194570ba8a438c5807e175dcf6b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3b0bf194570ba8a438c5807e175dcf6b columnFamilyName f1 2023-11-10 04:54:56,066 INFO [StoreOpener-3b0bf194570ba8a438c5807e175dcf6b-1] regionserver.HStore(310): Store=3b0bf194570ba8a438c5807e175dcf6b/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:54:56,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:54:56,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:54:56,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:54:56,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:54:56,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3b0bf194570ba8a438c5807e175dcf6b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60948908, jitterRate=-0.09179049730300903}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:54:56,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3b0bf194570ba8a438c5807e175dcf6b: 2023-11-10 04:54:56,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b., pid=73, masterSystemTime=1699592096056 2023-11-10 04:54:56,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:54:56,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:54:56,075 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=3b0bf194570ba8a438c5807e175dcf6b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:54:56,075 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1699592096075"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592096075"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592096075"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592096075"}]},"ts":"1699592096075"} 2023-11-10 04:54:56,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=73, resume processing ppid=72 2023-11-10 04:54:56,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 3b0bf194570ba8a438c5807e175dcf6b, server=jenkins-hbase4.apache.org,36419,1699592002485 in 171 msec 2023-11-10 04:54:56,078 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=72, resume processing ppid=71 2023-11-10 04:54:56,078 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=3b0bf194570ba8a438c5807e175dcf6b, ASSIGN in 326 msec 2023-11-10 04:54:56,079 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:54:56,079 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592096079"}]},"ts":"1699592096079"} 2023-11-10 04:54:56,080 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-11-10 04:54:56,082 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:54:56,082 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1580 sec 2023-11-10 04:54:57,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-10 04:54:57,031 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-11-10 04:54:57,031 DEBUG [Listener at localhost/35765] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-11-10 04:54:57,171 INFO [Listener at localhost/35765] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-11-10 04:54:57,171 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:57,171 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-11-10 04:54:57,171 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5fdb7e68 to 127.0.0.1:51169 2023-11-10 04:54:57,171 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:57,172 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2addf24f to 127.0.0.1:51169 2023-11-10 04:54:57,172 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:57,172 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:54:57,172 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x714066be to 127.0.0.1:51169 2023-11-10 04:54:57,172 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:54:57,176 INFO [Listener at localhost/35765] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-11-10 04:54:57,176 INFO [Listener at localhost/35765] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-11-10 04:54:57,177 INFO [Listener at localhost/35765] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-11-10 04:54:57,181 INFO [Listener at localhost/35765] regionserver.HRegion(2745): Flushing 3b0bf194570ba8a438c5807e175dcf6b 1/1 column families, dataSize=37 B heapSize=368 B 2023-11-10 04:54:57,593 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311101f64e5294a364d868ee0ae6097f1645b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202311101f64e5294a364d868ee0ae6097f1645b 2023-11-10 04:54:57,999 INFO [Listener at localhost/35765] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/.tmp/f1/ac8432dc3403408991cc8dc4517cf60e 2023-11-10 04:54:58,004 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/.tmp/f1/ac8432dc3403408991cc8dc4517cf60e as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e 2023-11-10 04:54:58,007 INFO [Listener at localhost/35765] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e, entries=1, sequenceid=5, filesize=4.9 K 2023-11-10 04:54:58,008 INFO [Listener at localhost/35765] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 3b0bf194570ba8a438c5807e175dcf6b in 828ms, sequenceid=5, compaction requested=false 2023-11-10 04:54:58,008 DEBUG [Listener at localhost/35765] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-11-10 04:54:58,009 DEBUG [Listener at localhost/35765] regionserver.HRegion(2446): Flush status journal for 3b0bf194570ba8a438c5807e175dcf6b: 2023-11-10 04:54:58,011 INFO [Listener at localhost/35765] regionserver.HRegion(2745): Flushing 3b0bf194570ba8a438c5807e175dcf6b 1/1 column families, dataSize=37 B heapSize=368 B 2023-11-10 04:54:58,212 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:54:58,421 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110d6a79cb8ed3b41959d1e181450cf69a9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231110d6a79cb8ed3b41959d1e181450cf69a9 2023-11-10 04:54:58,826 INFO [Listener at localhost/35765] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/.tmp/f1/7bc94519f7304656a866123b945cec64 2023-11-10 04:54:58,830 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/.tmp/f1/7bc94519f7304656a866123b945cec64 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64 2023-11-10 04:54:58,834 INFO [Listener at localhost/35765] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64, entries=1, sequenceid=9, filesize=4.9 K 2023-11-10 04:54:58,835 INFO [Listener at localhost/35765] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 3b0bf194570ba8a438c5807e175dcf6b in 824ms, sequenceid=9, compaction requested=false 2023-11-10 04:54:58,835 DEBUG [Listener at localhost/35765] regionserver.HRegion(2446): Flush status journal for 3b0bf194570ba8a438c5807e175dcf6b: 2023-11-10 04:54:58,836 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-11-10 04:54:58,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-10 04:54:58,838 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-10 04:54:58,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:58,839 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592698839 2023-11-10 04:54:58,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-10 04:54:58,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-11-10 04:54:58,839 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-11-10 04:54:58,840 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x1fad8314 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:54:58,844 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7488e82c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:54:58,845 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231110 for table='testGetAfterCompaction' and column='f1' 2023-11-10 04:54:58,887 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:58,888 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:58,938 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:58,939 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:58,989 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:58,989 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,040 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,040 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,091 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,091 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,142 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,142 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,193 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,193 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,244 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,244 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,295 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,295 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,346 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,346 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,397 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,397 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,448 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,448 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,499 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,499 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,549 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,550 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,600 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,600 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,651 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,651 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,669 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110985eb7fcc98340b8b4c06feb20ded4ce to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231110985eb7fcc98340b8b4c06feb20ded4ce 2023-11-10 04:54:59,671 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='testGetAfterCompaction' column='f1' 2023-11-10 04:54:59,685 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:54:59,686 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40750, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:54:59,692 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:54:59,694 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57882, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:54:59,702 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,702 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231110/f1/feb6f4b017c64703b194633dd26bd5ac first=Optional[r1] last=Optional[r2] 2023-11-10 04:54:59,702 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,702 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{f1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231110/f1/feb6f4b017c64703b194633dd26bd5ac}] 2023-11-10 04:54:59,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231110/f1/feb6f4b017c64703b194633dd26bd5ac for inclusion in 3b0bf194570ba8a438c5807e175dcf6b/f1 2023-11-10 04:54:59,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-11-10 04:54:59,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:54:59,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 3b0bf194570ba8a438c5807e175dcf6b: 2023-11-10 04:54:59,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231110/f1/feb6f4b017c64703b194633dd26bd5ac to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testGetAfterCompaction__mq94pinj40rbsnki07e4s2efghehmrn58b61ucvn11tkrjgem0403ubuv98fe5nb/f1/feb6f4b017c64703b194633dd26bd5ac 2023-11-10 04:54:59,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testGetAfterCompaction__mq94pinj40rbsnki07e4s2efghehmrn58b61ucvn11tkrjgem0403ubuv98fe5nb/f1/feb6f4b017c64703b194633dd26bd5ac as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_ 2023-11-10 04:54:59,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testGetAfterCompaction__mq94pinj40rbsnki07e4s2efghehmrn58b61ucvn11tkrjgem0403ubuv98fe5nb/f1/feb6f4b017c64703b194633dd26bd5ac into 3b0bf194570ba8a438c5807e175dcf6b/f1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_ - updating store file list. 2023-11-10 04:54:59,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_ into 3b0bf194570ba8a438c5807e175dcf6b/f1 2023-11-10 04:54:59,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testGetAfterCompaction__mq94pinj40rbsnki07e4s2efghehmrn58b61ucvn11tkrjgem0403ubuv98fe5nb/f1/feb6f4b017c64703b194633dd26bd5ac into 3b0bf194570ba8a438c5807e175dcf6b/f1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_) 2023-11-10 04:54:59,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testGetAfterCompaction__mq94pinj40rbsnki07e4s2efghehmrn58b61ucvn11tkrjgem0403ubuv98fe5nb/f1/feb6f4b017c64703b194633dd26bd5ac 2023-11-10 04:54:59,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): 3b0bf194570ba8a438c5807e175dcf6b/f1 is initiating minor compaction 2023-11-10 04:54:59,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5800a5fa; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:54:59,716 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3b0bf194570ba8a438c5807e175dcf6b/f1 in testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:54:59,717 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/.tmp, totalSize=10.0 K 2023-11-10 04:54:59,717 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 7bc94519f7304656a866123b945cec64, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-10 04:54:59,717 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting f0830960a55648b6a82a78a9bf354e15_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-10 04:54:59,718 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='testGetAfterCompaction' column='f1' 2023-11-10 04:54:59,719 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:54:59,721 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202311101f64e5294a364d868ee0ae6097f1645b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202311101f64e5294a364d868ee0ae6097f1645b 2023-11-10 04:54:59,722 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231110d6a79cb8ed3b41959d1e181450cf69a9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231110d6a79cb8ed3b41959d1e181450cf69a9 2023-11-10 04:54:59,722 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-11-10 04:54:59,722 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231110' 2023-11-10 04:54:59,723 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231110 for table='testGetAfterCompaction' and column='f1' 2023-11-10 04:54:59,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-11-10 04:54:59,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:54:59,730 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-10 04:54:59,731 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 893 msec 2023-11-10 04:54:59,732 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3b0bf194570ba8a438c5807e175dcf6b#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:54:59,753 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,753 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,753 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:54:59,756 DEBUG [Listener at localhost/35765] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-10 04:54:59,760 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40764, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-10 04:54:59,760 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36419] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:54:59,760 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-11-10 04:54:59,760 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-10 04:54:59,760 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36419] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-10 04:54:59,760 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36419] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. because compaction request was cancelled 2023-11-10 04:54:59,812 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,812 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,812 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:54:59,863 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,863 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,863 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:54:59,914 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,914 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,914 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:54:59,965 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,965 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:54:59,965 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,015 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,016 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,016 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,066 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,066 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,066 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,117 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,117 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,117 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,168 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,168 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,168 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,219 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,219 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,219 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,270 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,270 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,270 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,321 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,321 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,321 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,372 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,372 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,372 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,423 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,423 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,423 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,474 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,474 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,474 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,524 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592098826; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,525 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,525 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,545 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/.tmp/f1/e140ffec2356473eb9d7fd8f3977cc25 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/e140ffec2356473eb9d7fd8f3977cc25 2023-11-10 04:55:00,550 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 3b0bf194570ba8a438c5807e175dcf6b/f1 of 3b0bf194570ba8a438c5807e175dcf6b into e140ffec2356473eb9d7fd8f3977cc25(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:55:00,550 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3b0bf194570ba8a438c5807e175dcf6b: 2023-11-10 04:55:00,550 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1699592099716; duration=0sec 2023-11-10 04:55:00,550 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:55:00,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_] to archive 2023-11-10 04:55:00,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:00,575 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592100575; access_time=1699592098012; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,575 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,575 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/e140ffec2356473eb9d7fd8f3977cc25; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1699592100540; access_time=1699592099722; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/7bc94519f7304656a866123b945cec64 2023-11-10 04:55:00,575 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1699592099664; access_time=1699592098855; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-10 04:55:00,577 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/f0830960a55648b6a82a78a9bf354e15_SeqId_12_ 2023-11-10 04:55:00,626 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1699592097998; access_time=1699592097183; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,626 INFO [Listener at localhost/35765] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/e140ffec2356473eb9d7fd8f3977cc25; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1699592100540; access_time=1699592099722; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-10 04:55:00,675 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-10 04:55:00,741 INFO [Listener at localhost/35765] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-11-10 04:55:00,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-11-10 04:55:00,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-11-10 04:55:00,754 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592100753"}]},"ts":"1699592100753"} 2023-11-10 04:55:00,755 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-11-10 04:55:00,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-10 04:55:00,756 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-11-10 04:55:00,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=3b0bf194570ba8a438c5807e175dcf6b, UNASSIGN}] 2023-11-10 04:55:00,759 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=3b0bf194570ba8a438c5807e175dcf6b, UNASSIGN 2023-11-10 04:55:00,759 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=3b0bf194570ba8a438c5807e175dcf6b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:00,760 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1699592100759"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592100759"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592100759"}]},"ts":"1699592100759"} 2023-11-10 04:55:00,761 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 3b0bf194570ba8a438c5807e175dcf6b, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:00,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-10 04:55:00,912 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:00,912 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:55:00,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3b0bf194570ba8a438c5807e175dcf6b, disabling compactions & flushes 2023-11-10 04:55:00,912 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:55:00,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:55:00,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. after waiting 0 ms 2023-11-10 04:55:00,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:55:00,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-11-10 04:55:00,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-11-10 04:55:00,919 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b. 2023-11-10 04:55:00,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3b0bf194570ba8a438c5807e175dcf6b: 2023-11-10 04:55:00,920 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:55:00,921 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=3b0bf194570ba8a438c5807e175dcf6b, regionState=CLOSED 2023-11-10 04:55:00,921 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1699592100921"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592100921"}]},"ts":"1699592100921"} 2023-11-10 04:55:00,923 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=77, resume processing ppid=76 2023-11-10 04:55:00,923 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 3b0bf194570ba8a438c5807e175dcf6b, server=jenkins-hbase4.apache.org,36419,1699592002485 in 161 msec 2023-11-10 04:55:00,924 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=76, resume processing ppid=75 2023-11-10 04:55:00,924 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=3b0bf194570ba8a438c5807e175dcf6b, UNASSIGN in 165 msec 2023-11-10 04:55:00,925 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592100925"}]},"ts":"1699592100925"} 2023-11-10 04:55:00,926 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-11-10 04:55:00,929 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-11-10 04:55:00,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 182 msec 2023-11-10 04:55:01,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-10 04:55:01,057 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-11-10 04:55:01,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-11-10 04:55:01,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-11-10 04:55:01,069 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-10 04:55:01,070 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-10 04:55:01,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-11-10 04:55:01,073 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:55:01,075 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1, FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/recovered.edits] 2023-11-10 04:55:01,078 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/ac8432dc3403408991cc8dc4517cf60e 2023-11-10 04:55:01,079 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/e140ffec2356473eb9d7fd8f3977cc25 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/f1/e140ffec2356473eb9d7fd8f3977cc25 2023-11-10 04:55:01,081 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/recovered.edits/15.seqid to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b/recovered.edits/15.seqid 2023-11-10 04:55:01,082 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testGetAfterCompaction/3b0bf194570ba8a438c5807e175dcf6b 2023-11-10 04:55:01,082 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-11-10 04:55:01,082 DEBUG [PEWorker-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-11-10 04:55:01,082 DEBUG [PEWorker-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-11-10 04:55:01,085 DEBUG [PEWorker-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231110985eb7fcc98340b8b4c06feb20ded4ce to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231110985eb7fcc98340b8b4c06feb20ded4ce 2023-11-10 04:55:01,086 DEBUG [PEWorker-2] backup.HFileArchiver(596): Deleted hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-11-10 04:55:01,087 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-10 04:55:01,112 WARN [PEWorker-2] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-11-10 04:55:01,115 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-11-10 04:55:01,116 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-10 04:55:01,116 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-11-10 04:55:01,116 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1699592101116"}]},"ts":"9223372036854775807"} 2023-11-10 04:55:01,118 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-11-10 04:55:01,118 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3b0bf194570ba8a438c5807e175dcf6b, NAME => 'testGetAfterCompaction,,1699592094924.3b0bf194570ba8a438c5807e175dcf6b.', STARTKEY => '', ENDKEY => ''}] 2023-11-10 04:55:01,118 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-11-10 04:55:01,118 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1699592101118"}]},"ts":"9223372036854775807"} 2023-11-10 04:55:01,119 INFO [PEWorker-2] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-11-10 04:55:01,121 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-10 04:55:01,122 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 60 msec 2023-11-10 04:55:01,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-11-10 04:55:01,173 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-11-10 04:55:01,188 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=238 (was 244), OpenFileDescriptor=621 (was 653), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=124 (was 118) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=10740 (was 10794) 2023-11-10 04:55:01,200 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=238, OpenFileDescriptor=621, MaxFileDescriptor=60000, SystemLoadAverage=124, ProcessCount=167, AvailableMemoryMB=10739 2023-11-10 04:55:01,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-10 04:55:01,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-11-10 04:55:01,204 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:55:01,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-11-10 04:55:01,205 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:55:01,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-10 04:55:01,207 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,207 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,207 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:01,208 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723 empty. 2023-11-10 04:55:01,208 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8 empty. 2023-11-10 04:55:01,208 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6 empty. 2023-11-10 04:55:01,208 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,208 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,208 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:01,208 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-11-10 04:55:01,216 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-11-10 04:55:01,218 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 247092fa363b36c6fef420107a683723, NAME => 'testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:01,218 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 428db5f998f281799ebc91a16e0c51d8, NAME => 'testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:01,223 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => ee1662c8ccd6e1d2468c8603ba66a3a6, NAME => 'testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:01,236 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:01,236 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 247092fa363b36c6fef420107a683723, disabling compactions & flushes 2023-11-10 04:55:01,236 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:01,236 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:01,236 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. after waiting 0 ms 2023-11-10 04:55:01,236 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:01,236 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:01,236 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:01,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-10 04:55:01,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-10 04:55:01,643 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 428db5f998f281799ebc91a16e0c51d8, disabling compactions & flushes 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:01,644 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing ee1662c8ccd6e1d2468c8603ba66a3a6, disabling compactions & flushes 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:01,644 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. after waiting 0 ms 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:01,644 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. after waiting 0 ms 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:01,644 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:01,644 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:01,646 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:55:01,647 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592101647"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592101647"}]},"ts":"1699592101647"} 2023-11-10 04:55:01,647 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592101647"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592101647"}]},"ts":"1699592101647"} 2023-11-10 04:55:01,648 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592101647"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592101647"}]},"ts":"1699592101647"} 2023-11-10 04:55:01,651 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-10 04:55:01,652 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:55:01,652 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592101652"}]},"ts":"1699592101652"} 2023-11-10 04:55:01,653 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-11-10 04:55:01,659 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, ASSIGN}] 2023-11-10 04:55:01,660 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, ASSIGN 2023-11-10 04:55:01,661 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, ASSIGN 2023-11-10 04:55:01,661 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, ASSIGN 2023-11-10 04:55:01,662 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:01,662 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:01,662 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:01,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-10 04:55:01,813 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=428db5f998f281799ebc91a16e0c51d8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:01,813 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=ee1662c8ccd6e1d2468c8603ba66a3a6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:01,813 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=247092fa363b36c6fef420107a683723, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:01,814 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592101813"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592101813"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592101813"}]},"ts":"1699592101813"} 2023-11-10 04:55:01,814 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592101813"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592101813"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592101813"}]},"ts":"1699592101813"} 2023-11-10 04:55:01,814 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592101813"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592101813"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592101813"}]},"ts":"1699592101813"} 2023-11-10 04:55:01,815 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=83, ppid=81, state=RUNNABLE; OpenRegionProcedure 428db5f998f281799ebc91a16e0c51d8, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:01,816 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=84, ppid=80, state=RUNNABLE; OpenRegionProcedure 247092fa363b36c6fef420107a683723, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:01,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure ee1662c8ccd6e1d2468c8603ba66a3a6, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:01,969 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:01,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:01,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 247092fa363b36c6fef420107a683723, NAME => 'testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.', STARTKEY => '', ENDKEY => '1'} 2023-11-10 04:55:01,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:01,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,973 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,974 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1 2023-11-10 04:55:01,974 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1 2023-11-10 04:55:01,975 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 247092fa363b36c6fef420107a683723 columnFamilyName family1 2023-11-10 04:55:01,976 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(310): Store=247092fa363b36c6fef420107a683723/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:01,976 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,977 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2 2023-11-10 04:55:01,977 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2 2023-11-10 04:55:01,977 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 247092fa363b36c6fef420107a683723 columnFamilyName family2 2023-11-10 04:55:01,978 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(310): Store=247092fa363b36c6fef420107a683723/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:01,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:01,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 247092fa363b36c6fef420107a683723 2023-11-10 04:55:01,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:01,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 247092fa363b36c6fef420107a683723; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67601346, jitterRate=0.0073385536670684814}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:01,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:01,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723., pid=84, masterSystemTime=1699592101968 2023-11-10 04:55:01,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:01,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:01,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:01,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 428db5f998f281799ebc91a16e0c51d8, NAME => 'testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.', STARTKEY => '1', ENDKEY => '2'} 2023-11-10 04:55:01,986 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=247092fa363b36c6fef420107a683723, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:01,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:01,987 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592101986"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592101986"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592101986"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592101986"}]},"ts":"1699592101986"} 2023-11-10 04:55:01,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,988 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,989 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1 2023-11-10 04:55:01,989 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1 2023-11-10 04:55:01,989 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 428db5f998f281799ebc91a16e0c51d8 columnFamilyName family1 2023-11-10 04:55:01,989 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=84, resume processing ppid=80 2023-11-10 04:55:01,989 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=84, ppid=80, state=SUCCESS; OpenRegionProcedure 247092fa363b36c6fef420107a683723, server=jenkins-hbase4.apache.org,36419,1699592002485 in 172 msec 2023-11-10 04:55:01,990 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(310): Store=428db5f998f281799ebc91a16e0c51d8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:01,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, ASSIGN in 331 msec 2023-11-10 04:55:01,990 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,991 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2 2023-11-10 04:55:01,991 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2 2023-11-10 04:55:01,991 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 428db5f998f281799ebc91a16e0c51d8 columnFamilyName family2 2023-11-10 04:55:01,992 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(310): Store=428db5f998f281799ebc91a16e0c51d8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:01,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:01,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:01,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:01,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 428db5f998f281799ebc91a16e0c51d8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68100179, jitterRate=0.01477174460887909}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:01,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:01,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8., pid=83, masterSystemTime=1699592101968 2023-11-10 04:55:02,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:02,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:02,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:02,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee1662c8ccd6e1d2468c8603ba66a3a6, NAME => 'testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.', STARTKEY => '2', ENDKEY => ''} 2023-11-10 04:55:02,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:02,000 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=428db5f998f281799ebc91a16e0c51d8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:02,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:02,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:02,001 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592102000"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592102000"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592102000"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592102000"}]},"ts":"1699592102000"} 2023-11-10 04:55:02,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:02,002 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:02,003 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1 2023-11-10 04:55:02,003 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1 2023-11-10 04:55:02,004 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee1662c8ccd6e1d2468c8603ba66a3a6 columnFamilyName family1 2023-11-10 04:55:02,004 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=83, resume processing ppid=81 2023-11-10 04:55:02,004 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=83, ppid=81, state=SUCCESS; OpenRegionProcedure 428db5f998f281799ebc91a16e0c51d8, server=jenkins-hbase4.apache.org,36419,1699592002485 in 187 msec 2023-11-10 04:55:02,005 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(310): Store=ee1662c8ccd6e1d2468c8603ba66a3a6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:02,005 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, ASSIGN in 346 msec 2023-11-10 04:55:02,005 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:02,006 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2 2023-11-10 04:55:02,006 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2 2023-11-10 04:55:02,006 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee1662c8ccd6e1d2468c8603ba66a3a6 columnFamilyName family2 2023-11-10 04:55:02,007 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(310): Store=ee1662c8ccd6e1d2468c8603ba66a3a6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:02,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:02,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:02,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:02,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:02,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:02,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ee1662c8ccd6e1d2468c8603ba66a3a6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64589750, jitterRate=-0.037537723779678345}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:02,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:02,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6., pid=85, masterSystemTime=1699592101968 2023-11-10 04:55:02,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:02,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:02,018 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=ee1662c8ccd6e1d2468c8603ba66a3a6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:02,018 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592102018"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592102018"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592102018"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592102018"}]},"ts":"1699592102018"} 2023-11-10 04:55:02,023 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=85, resume processing ppid=82 2023-11-10 04:55:02,023 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure ee1662c8ccd6e1d2468c8603ba66a3a6, server=jenkins-hbase4.apache.org,36419,1699592002485 in 203 msec 2023-11-10 04:55:02,025 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=82, resume processing ppid=79 2023-11-10 04:55:02,025 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, ASSIGN in 364 msec 2023-11-10 04:55:02,025 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:55:02,026 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592102025"}]},"ts":"1699592102025"} 2023-11-10 04:55:02,027 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-11-10 04:55:02,031 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:55:02,032 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 828 msec 2023-11-10 04:55:02,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-10 04:55:02,309 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-11-10 04:55:02,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:02,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:02,318 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:02,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:02,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-10 04:55:02,322 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-10 04:55:02,322 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:02,322 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-10 04:55:02,322 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-10 04:55:02,322 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:02,322 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:02,324 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:02,324 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:02,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:02,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:02,324 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:02,324 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:02,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:02,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:02,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:02,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:02,325 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-10 04:55:02,325 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-10 04:55:02,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-10 04:55:02,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:02,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-10 04:55:02,326 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:02,326 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. started... 2023-11-10 04:55:02,326 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:02,326 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:02,326 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:02,326 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. started... 2023-11-10 04:55:02,326 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:02,326 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 428db5f998f281799ebc91a16e0c51d8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-10 04:55:02,327 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 247092fa363b36c6fef420107a683723 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-10 04:55:02,326 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. started... 2023-11-10 04:55:02,327 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing ee1662c8ccd6e1d2468c8603ba66a3a6 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-10 04:55:02,342 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110aa20f878baf844b0bdeca7e8694461f4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110aa20f878baf844b0bdeca7e8694461f4 2023-11-10 04:55:02,741 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311106b48e94e65554c4893ee5e1202c8783c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311106b48e94e65554c4893ee5e1202c8783c 2023-11-10 04:55:02,741 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111000281f5ebd834e8f9eb29aa07846bab0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111000281f5ebd834e8f9eb29aa07846bab0 2023-11-10 04:55:02,746 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/343bb832e9ac4876816536ca263980ac 2023-11-10 04:55:02,746 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/8a93989bdfc14dea990dbac25df2ee02 2023-11-10 04:55:02,747 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/335c1b58d71a4b0088492a1457fb1527 2023-11-10 04:55:03,168 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110f8196a9cfd674d4391e4aac41410be17 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110f8196a9cfd674d4391e4aac41410be17 2023-11-10 04:55:03,168 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110682a56e6bb8f493581f91f5b1006c728 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110682a56e6bb8f493581f91f5b1006c728 2023-11-10 04:55:03,170 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110aadf2e04b6f24e108e183dca05a37868 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110aadf2e04b6f24e108e183dca05a37868 2023-11-10 04:55:03,173 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/ae4b2c41494145ffab0e5a5e9c571e89 2023-11-10 04:55:03,177 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/8a93989bdfc14dea990dbac25df2ee02 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/8a93989bdfc14dea990dbac25df2ee02 2023-11-10 04:55:03,181 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/8a93989bdfc14dea990dbac25df2ee02, entries=4, sequenceid=5, filesize=5.4 K 2023-11-10 04:55:03,181 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/ae4b2c41494145ffab0e5a5e9c571e89 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ae4b2c41494145ffab0e5a5e9c571e89 2023-11-10 04:55:03,185 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ae4b2c41494145ffab0e5a5e9c571e89, entries=2, sequenceid=5, filesize=5.1 K 2023-11-10 04:55:03,186 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 428db5f998f281799ebc91a16e0c51d8 in 859ms, sequenceid=5, compaction requested=false 2023-11-10 04:55:03,186 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-11-10 04:55:03,186 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:03,186 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:03,186 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:03,573 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/097cfe0f69164ac28a93a645edd40a1e 2023-11-10 04:55:03,575 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/fdaea769dff24e2f87ae03469c289c42 2023-11-10 04:55:03,579 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/343bb832e9ac4876816536ca263980ac as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/343bb832e9ac4876816536ca263980ac 2023-11-10 04:55:03,581 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/335c1b58d71a4b0088492a1457fb1527 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/335c1b58d71a4b0088492a1457fb1527 2023-11-10 04:55:03,584 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/343bb832e9ac4876816536ca263980ac, entries=4, sequenceid=5, filesize=5.4 K 2023-11-10 04:55:03,584 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/097cfe0f69164ac28a93a645edd40a1e as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/097cfe0f69164ac28a93a645edd40a1e 2023-11-10 04:55:03,585 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/335c1b58d71a4b0088492a1457fb1527, entries=4, sequenceid=5, filesize=5.4 K 2023-11-10 04:55:03,586 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/fdaea769dff24e2f87ae03469c289c42 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/fdaea769dff24e2f87ae03469c289c42 2023-11-10 04:55:03,589 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/097cfe0f69164ac28a93a645edd40a1e, entries=2, sequenceid=5, filesize=5.1 K 2023-11-10 04:55:03,589 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/fdaea769dff24e2f87ae03469c289c42, entries=2, sequenceid=5, filesize=5.1 K 2023-11-10 04:55:03,590 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 247092fa363b36c6fef420107a683723 in 1263ms, sequenceid=5, compaction requested=false 2023-11-10 04:55:03,590 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:03,590 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:03,590 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:03,591 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for ee1662c8ccd6e1d2468c8603ba66a3a6 in 1264ms, sequenceid=5, compaction requested=false 2023-11-10 04:55:03,591 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:03,591 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:03,591 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:03,591 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:03,591 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,591 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-10 04:55:03,591 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-10 04:55:03,592 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,592 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:03,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:03,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:03,593 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:03,593 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:03,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:03,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:03,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:03,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:03,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-10 04:55:03,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5227eed[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:03,596 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-10 04:55:03,597 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:03,598 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:03,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:03,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:03,599 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-10 04:55:03,599 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-10 04:55:03,599 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:03,599 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,599 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:03,601 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-10 04:55:03,601 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:03,601 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,601 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-10 04:55:03,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:03,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:03,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:03,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:03,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:03,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:03,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:03,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:03,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-10 04:55:03,607 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-10 04:55:03,607 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:03,607 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:03,607 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-10 04:55:03,607 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:03,610 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:03,610 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:03,610 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:03,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:03,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:03,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:03,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:03,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:03,612 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:03,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:03,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:03,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:03,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:03,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:03,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:03,615 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:03,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:03,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,619 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,619 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:03,619 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:03,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-10 04:55:03,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:03,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-10 04:55:03,619 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:03,619 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:03,620 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:03,620 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:03,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:03,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:03,619 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,619 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:03,620 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:03,620 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:03,620 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:03,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:03,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:03,720 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:03,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:03,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:03,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:03,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:03,724 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:03,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:03,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-10 04:55:03,726 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-10 04:55:03,726 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:03,726 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-10 04:55:03,726 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-10 04:55:03,726 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:03,726 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:03,728 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:03,728 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:03,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:03,728 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:03,728 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:03,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:03,729 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:03,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:03,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:03,729 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-10 04:55:03,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-10 04:55:03,729 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-10 04:55:03,729 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-10 04:55:03,729 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:03,729 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-10 04:55:03,735 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:03,736 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. started... 2023-11-10 04:55:03,736 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:03,736 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 428db5f998f281799ebc91a16e0c51d8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-10 04:55:03,736 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:03,736 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:03,736 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:03,736 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. started... 2023-11-10 04:55:03,736 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. started... 2023-11-10 04:55:03,736 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing ee1662c8ccd6e1d2468c8603ba66a3a6 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-10 04:55:03,736 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 247092fa363b36c6fef420107a683723 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-10 04:55:04,165 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110efc6c47081fd44c791c6b6fe97a0558f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110efc6c47081fd44c791c6b6fe97a0558f 2023-11-10 04:55:04,165 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110d2c8fc62e00a4510a1d823cffe6d60a4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110d2c8fc62e00a4510a1d823cffe6d60a4 2023-11-10 04:55:04,170 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/f76926bedfc44a90aae90e75491d98dd 2023-11-10 04:55:04,171 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110b0ceb0985d4447449efb0ad15104d7fd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110b0ceb0985d4447449efb0ad15104d7fd 2023-11-10 04:55:04,175 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/1efd2e6cde824fe8afa345aaa743ec20 2023-11-10 04:55:04,571 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/c6696ef25fb345cbbf0b414f8500fedf 2023-11-10 04:55:04,583 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311109737724df754430a91cf28554ad175b8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311109737724df754430a91cf28554ad175b8 2023-11-10 04:55:04,588 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110e8a8ae26e34548f698d5b7198923d293 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110e8a8ae26e34548f698d5b7198923d293 2023-11-10 04:55:04,588 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/135a9c88def64bac9e94d078b28edbbf 2023-11-10 04:55:04,593 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/f76926bedfc44a90aae90e75491d98dd as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/f76926bedfc44a90aae90e75491d98dd 2023-11-10 04:55:04,597 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/f76926bedfc44a90aae90e75491d98dd, entries=4, sequenceid=9, filesize=5.4 K 2023-11-10 04:55:04,597 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/135a9c88def64bac9e94d078b28edbbf as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/135a9c88def64bac9e94d078b28edbbf 2023-11-10 04:55:04,600 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/135a9c88def64bac9e94d078b28edbbf, entries=2, sequenceid=9, filesize=5.1 K 2023-11-10 04:55:04,601 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 247092fa363b36c6fef420107a683723 in 865ms, sequenceid=9, compaction requested=false 2023-11-10 04:55:04,601 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:04,601 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:04,601 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:04,986 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110a68d06cb679f4f328510a8bc60678329 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110a68d06cb679f4f328510a8bc60678329 2023-11-10 04:55:04,993 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8 2023-11-10 04:55:04,997 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/1efd2e6cde824fe8afa345aaa743ec20 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/1efd2e6cde824fe8afa345aaa743ec20 2023-11-10 04:55:05,001 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/1efd2e6cde824fe8afa345aaa743ec20, entries=4, sequenceid=9, filesize=5.4 K 2023-11-10 04:55:05,002 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8 2023-11-10 04:55:05,006 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8, entries=2, sequenceid=9, filesize=5.1 K 2023-11-10 04:55:05,006 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for ee1662c8ccd6e1d2468c8603ba66a3a6 in 1270ms, sequenceid=9, compaction requested=false 2023-11-10 04:55:05,007 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:05,007 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:05,007 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:05,392 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/ad709bef78f04200a7e5a19bfc072ba1 2023-11-10 04:55:05,396 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/c6696ef25fb345cbbf0b414f8500fedf as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/c6696ef25fb345cbbf0b414f8500fedf 2023-11-10 04:55:05,400 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/c6696ef25fb345cbbf0b414f8500fedf, entries=4, sequenceid=9, filesize=5.4 K 2023-11-10 04:55:05,400 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/ad709bef78f04200a7e5a19bfc072ba1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ad709bef78f04200a7e5a19bfc072ba1 2023-11-10 04:55:05,404 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ad709bef78f04200a7e5a19bfc072ba1, entries=2, sequenceid=9, filesize=5.1 K 2023-11-10 04:55:05,405 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 428db5f998f281799ebc91a16e0c51d8 in 1669ms, sequenceid=9, compaction requested=false 2023-11-10 04:55:05,405 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:05,405 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:05,405 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:05,405 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:05,405 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,405 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-10 04:55:05,405 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-10 04:55:05,412 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,412 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:05,412 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:05,412 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:05,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-10 04:55:05,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d9224d4[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:05,414 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-10 04:55:05,414 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:05,415 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:05,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:05,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:05,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-10 04:55:05,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-10 04:55:05,415 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:05,415 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,415 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:05,417 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-10 04:55:05,417 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:05,417 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,417 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-10 04:55:05,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:05,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:05,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:05,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:05,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:05,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-10 04:55:05,420 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-10 04:55:05,420 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:05,420 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:05,420 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-10 04:55:05,420 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:05,422 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:05,422 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:05,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:05,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:05,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:05,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:05,422 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:05,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:05,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:05,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:05,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:05,422 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:05,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:05,425 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:05,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:05,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,428 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,428 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:05,428 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:05,428 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:05,429 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:05,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:05,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:05,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-10 04:55:05,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:05,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-10 04:55:05,428 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:05,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:05,429 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:05,429 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:05,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:05,429 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,430 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:05,430 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:05,430 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:05,530 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:05,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:05,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:05,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:05,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:05,535 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:05,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:05,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-10 04:55:05,541 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-10 04:55:05,541 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:05,541 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-10 04:55:05,541 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-10 04:55:05,542 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:05,542 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:05,543 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:05,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:05,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:05,543 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:05,544 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:05,544 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:05,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:05,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:05,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:05,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-10 04:55:05,553 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-10 04:55:05,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-10 04:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-10 04:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-10 04:55:05,553 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:05,553 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. started... 2023-11-10 04:55:05,553 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:05,553 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. started... 2023-11-10 04:55:05,553 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 428db5f998f281799ebc91a16e0c51d8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-10 04:55:05,553 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 247092fa363b36c6fef420107a683723 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-10 04:55:05,554 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:05,554 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:05,554 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:05,554 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. started... 2023-11-10 04:55:05,554 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing ee1662c8ccd6e1d2468c8603ba66a3a6 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-10 04:55:05,598 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311108a9f8f2e160e4ae297ff5d6758f35d78 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311108a9f8f2e160e4ae297ff5d6758f35d78 2023-11-10 04:55:05,601 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110a49554fc563041f8a8a1665675a6885b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110a49554fc563041f8a8a1665675a6885b 2023-11-10 04:55:05,988 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110ed18e5da3f2348169787daf780e530cd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110ed18e5da3f2348169787daf780e530cd 2023-11-10 04:55:06,027 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/9a7ec7a970b54c7d84a57d195c12f5d5 2023-11-10 04:55:06,029 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/b2d95126b7b4496c920348e7cc43b2be 2023-11-10 04:55:06,393 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/54d22b6b43374e0ab599ff9a0a26dabc 2023-11-10 04:55:06,421 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:55:06,445 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023111048cf145946e0415dbff474330468d120 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111048cf145946e0415dbff474330468d120 2023-11-10 04:55:06,445 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110a7700e7ca441486fb327e27f060c222e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110a7700e7ca441486fb327e27f060c222e 2023-11-10 04:55:06,806 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110383e4615c7b74a309ccb47b415522473 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110383e4615c7b74a309ccb47b415522473 2023-11-10 04:55:06,851 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/230ea277da404cf39dc02f7f034f7fd7 2023-11-10 04:55:06,854 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/b2d95126b7b4496c920348e7cc43b2be as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/b2d95126b7b4496c920348e7cc43b2be 2023-11-10 04:55:06,858 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/b2d95126b7b4496c920348e7cc43b2be, entries=4, sequenceid=13, filesize=5.4 K 2023-11-10 04:55:06,858 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/230ea277da404cf39dc02f7f034f7fd7 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/230ea277da404cf39dc02f7f034f7fd7 2023-11-10 04:55:06,861 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/230ea277da404cf39dc02f7f034f7fd7, entries=2, sequenceid=13, filesize=5.1 K 2023-11-10 04:55:06,862 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/a63356bdf54c4d7b9c03c16621133a28 2023-11-10 04:55:06,862 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 247092fa363b36c6fef420107a683723 in 1309ms, sequenceid=13, compaction requested=true 2023-11-10 04:55:06,862 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:06,862 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:06,863 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:06,866 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/9a7ec7a970b54c7d84a57d195c12f5d5 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/9a7ec7a970b54c7d84a57d195c12f5d5 2023-11-10 04:55:06,870 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/9a7ec7a970b54c7d84a57d195c12f5d5, entries=4, sequenceid=13, filesize=5.4 K 2023-11-10 04:55:06,871 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/a63356bdf54c4d7b9c03c16621133a28 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/a63356bdf54c4d7b9c03c16621133a28 2023-11-10 04:55:06,875 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/a63356bdf54c4d7b9c03c16621133a28, entries=2, sequenceid=13, filesize=5.1 K 2023-11-10 04:55:06,876 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for ee1662c8ccd6e1d2468c8603ba66a3a6 in 1322ms, sequenceid=13, compaction requested=true 2023-11-10 04:55:06,876 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:06,876 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:06,876 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:07,211 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/4ac76495f51f4d65856ae576d60f9971 2023-11-10 04:55:07,218 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/54d22b6b43374e0ab599ff9a0a26dabc as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/54d22b6b43374e0ab599ff9a0a26dabc 2023-11-10 04:55:07,221 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/54d22b6b43374e0ab599ff9a0a26dabc, entries=4, sequenceid=13, filesize=5.4 K 2023-11-10 04:55:07,222 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/4ac76495f51f4d65856ae576d60f9971 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4ac76495f51f4d65856ae576d60f9971 2023-11-10 04:55:07,226 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4ac76495f51f4d65856ae576d60f9971, entries=2, sequenceid=13, filesize=5.1 K 2023-11-10 04:55:07,227 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 428db5f998f281799ebc91a16e0c51d8 in 1674ms, sequenceid=13, compaction requested=true 2023-11-10 04:55:07,227 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:07,227 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:07,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:07,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:07,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-10 04:55:07,227 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-10 04:55:07,230 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:07,231 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:07,231 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:07,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:07,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-10 04:55:07,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28cd15f1[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:07,232 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-10 04:55:07,232 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:07,233 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:07,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:07,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:07,234 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-10 04:55:07,234 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-10 04:55:07,234 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:07,234 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,234 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:07,235 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-10 04:55:07,235 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,235 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:07,235 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-10 04:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:07,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:07,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:07,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-10 04:55:07,238 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-10 04:55:07,238 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:07,238 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:07,238 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-10 04:55:07,238 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:07,241 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:07,241 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:07,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:07,241 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:07,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:07,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,243 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:07,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:07,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,247 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,247 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:07,247 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:07,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:07,247 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:07,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-10 04:55:07,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:07,247 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:07,247 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:07,247 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:07,248 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:07,248 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:07,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:07,248 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:07,248 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:07,248 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:07,348 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:07,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:07,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:07,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:07,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:07,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:07,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:07,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-10 04:55:07,354 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-10 04:55:07,355 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:07,355 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-10 04:55:07,355 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-10 04:55:07,355 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:07,355 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:07,357 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,357 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:07,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:07,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:07,357 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:07,357 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:07,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:07,358 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:07,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:07,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:07,358 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-10 04:55:07,358 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-10 04:55:07,358 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-10 04:55:07,358 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-10 04:55:07,358 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:07,358 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-10 04:55:07,359 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:07,359 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. started... 2023-11-10 04:55:07,359 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:07,359 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. started... 2023-11-10 04:55:07,359 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 428db5f998f281799ebc91a16e0c51d8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-10 04:55:07,359 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:07,359 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:07,359 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:07,359 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. started... 2023-11-10 04:55:07,359 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 247092fa363b36c6fef420107a683723 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-10 04:55:07,359 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing ee1662c8ccd6e1d2468c8603ba66a3a6 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-10 04:55:07,378 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110778980a6a03548eaa0eb0261594b0c47 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110778980a6a03548eaa0eb0261594b0c47 2023-11-10 04:55:07,780 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110cefd0f548db540a78c386a0d2a56fde4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110cefd0f548db540a78c386a0d2a56fde4 2023-11-10 04:55:07,781 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110b0d110932bae45359e8a228021cd70f1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110b0d110932bae45359e8a228021cd70f1 2023-11-10 04:55:07,785 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/18fa4d39f76c454b84c3556bc005dec3 2023-11-10 04:55:08,205 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/61ea0ea6261a4724adaccf6dc92aa8fa 2023-11-10 04:55:08,205 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/0563518c4e2d4880b8cc81f80d202363 2023-11-10 04:55:08,212 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023111033e0034756e34f0db93bbc10f36c3355 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111033e0034756e34f0db93bbc10f36c3355 2023-11-10 04:55:08,227 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111003c9829c1ab445fbb1840b9e244a99dc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111003c9829c1ab445fbb1840b9e244a99dc 2023-11-10 04:55:08,621 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110f5ff6c793ec14b4292dea798e6a487bd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110f5ff6c793ec14b4292dea798e6a487bd 2023-11-10 04:55:08,622 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/c07b5707fdaa439788a90b8039e6f558 2023-11-10 04:55:08,625 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/a20efa896262435186ccb4afda7dcaa0 2023-11-10 04:55:08,627 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/18fa4d39f76c454b84c3556bc005dec3 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/18fa4d39f76c454b84c3556bc005dec3 2023-11-10 04:55:08,629 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/0563518c4e2d4880b8cc81f80d202363 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0563518c4e2d4880b8cc81f80d202363 2023-11-10 04:55:08,631 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/18fa4d39f76c454b84c3556bc005dec3, entries=4, sequenceid=17, filesize=5.4 K 2023-11-10 04:55:08,631 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/3dfe247256a54b92a5549e8aa06c7a79 2023-11-10 04:55:08,631 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/c07b5707fdaa439788a90b8039e6f558 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c07b5707fdaa439788a90b8039e6f558 2023-11-10 04:55:08,633 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0563518c4e2d4880b8cc81f80d202363, entries=4, sequenceid=17, filesize=5.4 K 2023-11-10 04:55:08,634 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/a20efa896262435186ccb4afda7dcaa0 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/a20efa896262435186ccb4afda7dcaa0 2023-11-10 04:55:08,638 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/61ea0ea6261a4724adaccf6dc92aa8fa as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/61ea0ea6261a4724adaccf6dc92aa8fa 2023-11-10 04:55:08,638 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c07b5707fdaa439788a90b8039e6f558, entries=2, sequenceid=17, filesize=5.1 K 2023-11-10 04:55:08,639 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 247092fa363b36c6fef420107a683723 in 1280ms, sequenceid=17, compaction requested=true 2023-11-10 04:55:08,639 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:08,639 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:08,639 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:08,641 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/a20efa896262435186ccb4afda7dcaa0, entries=2, sequenceid=17, filesize=5.1 K 2023-11-10 04:55:08,641 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 428db5f998f281799ebc91a16e0c51d8 in 1282ms, sequenceid=17, compaction requested=true 2023-11-10 04:55:08,642 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:08,642 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:08,642 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:08,642 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/61ea0ea6261a4724adaccf6dc92aa8fa, entries=4, sequenceid=17, filesize=5.4 K 2023-11-10 04:55:08,643 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/3dfe247256a54b92a5549e8aa06c7a79 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/3dfe247256a54b92a5549e8aa06c7a79 2023-11-10 04:55:08,646 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/3dfe247256a54b92a5549e8aa06c7a79, entries=2, sequenceid=17, filesize=5.1 K 2023-11-10 04:55:08,647 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for ee1662c8ccd6e1d2468c8603ba66a3a6 in 1288ms, sequenceid=17, compaction requested=true 2023-11-10 04:55:08,647 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:08,647 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-10 04:55:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-10 04:55:08,649 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:08,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:08,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:08,649 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:08,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:08,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:08,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:08,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:08,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-10 04:55:08,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ecc504[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:08,651 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-10 04:55:08,651 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:08,652 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:08,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:08,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:08,652 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-10 04:55:08,652 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-10 04:55:08,652 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:08,652 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,652 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:08,653 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-10 04:55:08,653 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,653 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:08,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,653 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-10 04:55:08,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:08,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:08,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:08,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:08,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:08,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:08,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:08,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:08,656 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-10 04:55:08,656 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:08,656 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:08,656 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-10 04:55:08,656 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:08,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-10 04:55:08,658 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:08,658 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:08,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:08,658 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:08,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:08,659 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:08,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:08,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:08,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,660 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:08,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:08,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:08,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:08,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-10 04:55:08,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:08,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:08,663 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:08,663 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:08,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:08,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:08,763 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:08,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:08,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-10 04:55:08,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1699592108775 (current time:1699592108775). 2023-11-10 04:55:08,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-11-10 04:55:08,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] zookeeper.ReadOnlyZKClient(139): Connect 0x4aab2b26 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:55:08,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7132eaf5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:55:08,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:55:08,790 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54354, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:55:08,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4aab2b26 to 127.0.0.1:51169 2023-11-10 04:55:08,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:55:08,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-11-10 04:55:08,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(684): Table enabled, starting distributed snapshots for { ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-10 04:55:08,802 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot, current working dir path: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/snaptb-1699592108764 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotEnabledTable(SnapshotManager.java:559) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:687) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-11-10 04:55:08,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-10 04:55:08,811 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-10 04:55:08,812 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1699592708812 2023-11-10 04:55:08,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(689): Started snapshot: { ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-10 04:55:08,812 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1699592108764 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-11-10 04:55:08,813 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-10 04:55:08,814 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1028): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-10 04:55:08,814 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-11-10 04:55:08,815 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-10 04:55:08,815 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1699592708815 2023-11-10 04:55:08,818 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:08,818 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-11-10 04:55:08,918 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:08,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:08,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-10 04:55:08,922 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-11-10 04:55:09,122 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:09,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:09,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-10 04:55:09,124 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-11-10 04:55:09,222 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1699592108764 2023-11-10 04:55:09,223 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1699592108764' 2023-11-10 04:55:09,223 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-11-10 04:55:09,223 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1699592108764' starting 'acquire' 2023-11-10 04:55:09,223 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1699592108764', kicking off acquire phase on members. 2023-11-10 04:55:09,223 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1699592108764 2023-11-10 04:55:09,223 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1699592108764 2023-11-10 04:55:09,225 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,225 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-11-10 04:55:09,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-11-10 04:55:09,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-10 04:55:09,225 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,225 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:09,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1699592108764 2023-11-10 04:55:09,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1699592108764 2023-11-10 04:55:09,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-11-10 04:55:09,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1699592108764 2023-11-10 04:55:09,226 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1699592108764 from table testCompactionWithHFileLink type FLUSH 2023-11-10 04:55:09,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1699592108764 2023-11-10 04:55:09,229 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1699592108764' with timeout 300000ms 2023-11-10 04:55:09,229 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-11-10 04:55:09,229 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1699592108764' starting 'acquire' stage 2023-11-10 04:55:09,229 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1699592108764' locally acquired 2023-11-10 04:55:09,229 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (snaptb-1699592108764) in zk 2023-11-10 04:55:09,231 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,231 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1699592108764 2023-11-10 04:55:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-10 04:55:09,231 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1699592108764 2023-11-10 04:55:09,231 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1699592108764' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:09,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:09,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1699592108764 2023-11-10 04:55:09,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:09,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'snaptb-1699592108764' on coordinator 2023-11-10 04:55:09,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11af0602[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:09,232 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1699592108764' starting 'in-barrier' execution. 2023-11-10 04:55:09,232 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1699592108764 2023-11-10 04:55:09,234 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1699592108764 2023-11-10 04:55:09,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1699592108764 2023-11-10 04:55:09,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1699592108764 2023-11-10 04:55:09,234 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1699592108764' received 'reached' from coordinator. 2023-11-10 04:55:09,235 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,235 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:09,235 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:09,235 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:09,236 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-11-10 04:55:09,236 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:09,236 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-11-10 04:55:09,236 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. started... 2023-11-10 04:55:09,236 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. started... 2023-11-10 04:55:09,236 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. started... 2023-11-10 04:55:09,236 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:09,236 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:09,236 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:09,237 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.' region-info for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,237 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.' region-info for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,237 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.' region-info for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,242 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-10 04:55:09,242 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-10 04:55:09,242 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-10 04:55:09,245 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/335c1b58d71a4b0088492a1457fb1527, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/1efd2e6cde824fe8afa345aaa743ec20, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/9a7ec7a970b54c7d84a57d195c12f5d5, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/61ea0ea6261a4724adaccf6dc92aa8fa] hfiles 2023-11-10 04:55:09,245 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/8a93989bdfc14dea990dbac25df2ee02, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/c6696ef25fb345cbbf0b414f8500fedf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/54d22b6b43374e0ab599ff9a0a26dabc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0563518c4e2d4880b8cc81f80d202363] hfiles 2023-11-10 04:55:09,245 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/335c1b58d71a4b0088492a1457fb1527 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,245 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/343bb832e9ac4876816536ca263980ac, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/f76926bedfc44a90aae90e75491d98dd, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/b2d95126b7b4496c920348e7cc43b2be, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/18fa4d39f76c454b84c3556bc005dec3] hfiles 2023-11-10 04:55:09,245 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/8a93989bdfc14dea990dbac25df2ee02 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,246 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/343bb832e9ac4876816536ca263980ac for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/c6696ef25fb345cbbf0b414f8500fedf for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/f76926bedfc44a90aae90e75491d98dd for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/1efd2e6cde824fe8afa345aaa743ec20 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/b2d95126b7b4496c920348e7cc43b2be for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/54d22b6b43374e0ab599ff9a0a26dabc for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/18fa4d39f76c454b84c3556bc005dec3 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/9a7ec7a970b54c7d84a57d195c12f5d5 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0563518c4e2d4880b8cc81f80d202363 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/61ea0ea6261a4724adaccf6dc92aa8fa for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ae4b2c41494145ffab0e5a5e9c571e89, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ad709bef78f04200a7e5a19bfc072ba1, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4ac76495f51f4d65856ae576d60f9971, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/a20efa896262435186ccb4afda7dcaa0] hfiles 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ae4b2c41494145ffab0e5a5e9c571e89 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/fdaea769dff24e2f87ae03469c289c42, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/a63356bdf54c4d7b9c03c16621133a28, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/3dfe247256a54b92a5549e8aa06c7a79] hfiles 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/fdaea769dff24e2f87ae03469c289c42 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/097cfe0f69164ac28a93a645edd40a1e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/135a9c88def64bac9e94d078b28edbbf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/230ea277da404cf39dc02f7f034f7fd7, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c07b5707fdaa439788a90b8039e6f558] hfiles 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ad709bef78f04200a7e5a19bfc072ba1 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/a63356bdf54c4d7b9c03c16621133a28 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/097cfe0f69164ac28a93a645edd40a1e for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/3dfe247256a54b92a5549e8aa06c7a79 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4ac76495f51f4d65856ae576d60f9971 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/135a9c88def64bac9e94d078b28edbbf for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,249 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/a20efa896262435186ccb4afda7dcaa0 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,250 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/230ea277da404cf39dc02f7f034f7fd7 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,250 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c07b5707fdaa439788a90b8039e6f558 for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,424 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:09,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:09,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-10 04:55:09,425 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-11-10 04:55:09,656 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. completed. 2023-11-10 04:55:09,656 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:09,656 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-11-10 04:55:09,658 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. completed. 2023-11-10 04:55:09,658 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:09,658 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-11-10 04:55:09,663 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. completed. 2023-11-10 04:55:09,663 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-11-10 04:55:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-11-10 04:55:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1699592108764' locally completed 2023-11-10 04:55:09,663 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1699592108764' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:09,665 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1699592108764' has notified controller of completion 2023-11-10 04:55:09,665 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,665 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:09,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,665 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1699592108764' completed. 2023-11-10 04:55:09,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:09,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-10 04:55:09,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:09,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:09,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1699592108764 2023-11-10 04:55:09,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:09,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1699592108764 2023-11-10 04:55:09,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1699592108764' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:09,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'snaptb-1699592108764', counting down latch. Waiting for 0 more 2023-11-10 04:55:09,667 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1699592108764' execution completed 2023-11-10 04:55:09,667 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:09,667 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:09,667 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1699592108764 2023-11-10 04:55:09,667 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1699592108764including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-11-10 04:55:09,669 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1699592108764 2023-11-10 04:55:09,669 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1699592108764 2023-11-10 04:55:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1699592108764 2023-11-10 04:55:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:09,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1699592108764 2023-11-10 04:55:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1699592108764 2023-11-10 04:55:09,669 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-11-10 04:55:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-10 04:55:09,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-11-10 04:55:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-10 04:55:09,669 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1699592108764 2023-11-10 04:55:09,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1699592108764 2023-11-10 04:55:09,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:09,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1699592108764 2023-11-10 04:55:09,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,670 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1699592108764 2023-11-10 04:55:09,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,672 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,672 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-11-10 04:55:09,672 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1699592108764 2023-11-10 04:55:09,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-11-10 04:55:09,672 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:09,672 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1699592108764 2023-11-10 04:55:09,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-10 04:55:09,672 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-11-10 04:55:09,672 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1699592108764 2023-11-10 04:55:09,673 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1699592108764 because node does not exist (not an error) 2023-11-10 04:55:09,673 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-11-10 04:55:09,673 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1699592108764/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:09,673 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1699592108764 2023-11-10 04:55:09,673 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1699592108764 2023-11-10 04:55:09,673 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1699592108764 2023-11-10 04:55:09,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-11-10 04:55:09,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-10 04:55:09,673 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1699592108764 2023-11-10 04:55:09,673 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110aa20f878baf844b0bdeca7e8694461f4, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110b0d110932bae45359e8a228021cd70f1, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110d2c8fc62e00a4510a1d823cffe6d60a4, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110ed18e5da3f2348169787daf780e530cd, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111000281f5ebd834e8f9eb29aa07846bab0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110a49554fc563041f8a8a1665675a6885b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110b0ceb0985d4447449efb0ad15104d7fd, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110cefd0f548db540a78c386a0d2a56fde4, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311106b48e94e65554c4893ee5e1202c8783c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110778980a6a03548eaa0eb0261594b0c47, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311108a9f8f2e160e4ae297ff5d6758f35d78, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110efc6c47081fd44c791c6b6fe97a0558f] mob files 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110aa20f878baf844b0bdeca7e8694461f4 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110b0d110932bae45359e8a228021cd70f1 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110d2c8fc62e00a4510a1d823cffe6d60a4 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110ed18e5da3f2348169787daf780e530cd 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111000281f5ebd834e8f9eb29aa07846bab0 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110a49554fc563041f8a8a1665675a6885b 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110b0ceb0985d4447449efb0ad15104d7fd 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110cefd0f548db540a78c386a0d2a56fde4 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311106b48e94e65554c4893ee5e1202c8783c 2023-11-10 04:55:09,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110778980a6a03548eaa0eb0261594b0c47 2023-11-10 04:55:09,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311108a9f8f2e160e4ae297ff5d6758f35d78 2023-11-10 04:55:09,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110efc6c47081fd44c791c6b6fe97a0558f 2023-11-10 04:55:09,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110383e4615c7b74a309ccb47b415522473, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110682a56e6bb8f493581f91f5b1006c728, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110a68d06cb679f4f328510a8bc60678329, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110f5ff6c793ec14b4292dea798e6a487bd, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111003c9829c1ab445fbb1840b9e244a99dc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110a7700e7ca441486fb327e27f060c222e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110aadf2e04b6f24e108e183dca05a37868, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110e8a8ae26e34548f698d5b7198923d293, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111033e0034756e34f0db93bbc10f36c3355, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111048cf145946e0415dbff474330468d120, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311109737724df754430a91cf28554ad175b8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110f8196a9cfd674d4391e4aac41410be17] mob files 2023-11-10 04:55:09,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110383e4615c7b74a309ccb47b415522473 2023-11-10 04:55:09,675 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110682a56e6bb8f493581f91f5b1006c728 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110a68d06cb679f4f328510a8bc60678329 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110f5ff6c793ec14b4292dea798e6a487bd 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111003c9829c1ab445fbb1840b9e244a99dc 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110a7700e7ca441486fb327e27f060c222e 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110aadf2e04b6f24e108e183dca05a37868 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110e8a8ae26e34548f698d5b7198923d293 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111033e0034756e34f0db93bbc10f36c3355 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111048cf145946e0415dbff474330468d120 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311109737724df754430a91cf28554ad175b8 2023-11-10 04:55:09,676 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110f8196a9cfd674d4391e4aac41410be17 2023-11-10 04:55:09,925 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:09,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:09,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-10 04:55:09,926 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-11-10 04:55:10,081 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1699592108764 2023-11-10 04:55:10,083 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/snaptb-1699592108764 2023-11-10 04:55:10,526 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/snaptb-1699592108764 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/snaptb-1699592108764 2023-11-10 04:55:10,527 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1699592108764 of table testCompactionWithHFileLink completed 2023-11-10 04:55:10,527 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/snaptb-1699592108764 2023-11-10 04:55:10,528 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/snaptb-1699592108764 2023-11-10 04:55:10,528 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1699592108764 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1699592108812Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1699592110081 (+1269 ms)Consolidate snapshot: snaptb-1699592108764 at 1699592110081Loading Region manifests for snaptb-1699592108764 at 1699592110081Writing data manifest for snaptb-1699592108764 at 1699592110092 (+11 ms)Verifying snapshot: snaptb-1699592108764 at 1699592110500 (+408 ms)Snapshot snaptb-1699592108764 of table testCompactionWithHFileLink completed at 1699592110527 (+27 ms) 2023-11-10 04:55:10,529 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-10 04:55:10,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.7150 sec 2023-11-10 04:55:10,926 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:10,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:10,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(403): Snapshot '{ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-11-10 04:55:10,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:10,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:10,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-10 04:55:10,966 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-10 04:55:10,966 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:10,966 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-10 04:55:10,966 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-10 04:55:10,967 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:10,967 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:10,968 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:10,968 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:10,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:10,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:10,968 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:10,968 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:10,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:10,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:10,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:10,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:10,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-10 04:55:10,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-10 04:55:10,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-10 04:55:10,970 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-10 04:55:10,970 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:10,970 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-10 04:55:10,970 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:10,970 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. started... 2023-11-10 04:55:10,970 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 428db5f998f281799ebc91a16e0c51d8 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-10 04:55:10,970 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:10,970 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. started... 2023-11-10 04:55:10,970 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:10,971 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 247092fa363b36c6fef420107a683723 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-10 04:55:10,971 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:10,971 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:10,971 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. started... 2023-11-10 04:55:10,971 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing ee1662c8ccd6e1d2468c8603ba66a3a6 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-10 04:55:11,401 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/7e24d760b113484bafd056ef97e2e2ab 2023-11-10 04:55:11,404 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e24d760b113484bafd056ef97e2e2ab 2023-11-10 04:55:11,804 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/5fd5e1738faa40968a01401ae1384146 2023-11-10 04:55:11,804 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/5a8e17569fb14fbfaa1e22bacad1a2d5 2023-11-10 04:55:11,810 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fd5e1738faa40968a01401ae1384146 2023-11-10 04:55:11,810 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a8e17569fb14fbfaa1e22bacad1a2d5 2023-11-10 04:55:11,816 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/c8f06fdd459341d7bd5b8ee8427685f9 2023-11-10 04:55:11,820 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8f06fdd459341d7bd5b8ee8427685f9 2023-11-10 04:55:11,821 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/7e24d760b113484bafd056ef97e2e2ab as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/7e24d760b113484bafd056ef97e2e2ab 2023-11-10 04:55:11,824 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e24d760b113484bafd056ef97e2e2ab 2023-11-10 04:55:11,824 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/7e24d760b113484bafd056ef97e2e2ab, entries=3, sequenceid=24, filesize=5.0 K 2023-11-10 04:55:11,825 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/c8f06fdd459341d7bd5b8ee8427685f9 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c8f06fdd459341d7bd5b8ee8427685f9 2023-11-10 04:55:11,829 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8f06fdd459341d7bd5b8ee8427685f9 2023-11-10 04:55:11,829 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c8f06fdd459341d7bd5b8ee8427685f9, entries=1, sequenceid=24, filesize=5.0 K 2023-11-10 04:55:11,830 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 247092fa363b36c6fef420107a683723 in 859ms, sequenceid=24, compaction requested=true 2023-11-10 04:55:11,830 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:11,830 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:11,830 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:12,632 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/dead17dad2744895bec8fa605f793efe 2023-11-10 04:55:12,636 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dead17dad2744895bec8fa605f793efe 2023-11-10 04:55:12,637 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/5fd5e1738faa40968a01401ae1384146 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/5fd5e1738faa40968a01401ae1384146 2023-11-10 04:55:12,639 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/1974f7d317f64c329bae420ca51b136e 2023-11-10 04:55:12,642 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fd5e1738faa40968a01401ae1384146 2023-11-10 04:55:12,642 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/5fd5e1738faa40968a01401ae1384146, entries=3, sequenceid=24, filesize=5.0 K 2023-11-10 04:55:12,643 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/dead17dad2744895bec8fa605f793efe as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/dead17dad2744895bec8fa605f793efe 2023-11-10 04:55:12,644 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1974f7d317f64c329bae420ca51b136e 2023-11-10 04:55:12,645 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/5a8e17569fb14fbfaa1e22bacad1a2d5 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/5a8e17569fb14fbfaa1e22bacad1a2d5 2023-11-10 04:55:12,647 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dead17dad2744895bec8fa605f793efe 2023-11-10 04:55:12,647 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/dead17dad2744895bec8fa605f793efe, entries=1, sequenceid=24, filesize=5.0 K 2023-11-10 04:55:12,648 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 428db5f998f281799ebc91a16e0c51d8 in 1678ms, sequenceid=24, compaction requested=true 2023-11-10 04:55:12,648 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:12,648 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:12,648 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:12,649 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a8e17569fb14fbfaa1e22bacad1a2d5 2023-11-10 04:55:12,650 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/5a8e17569fb14fbfaa1e22bacad1a2d5, entries=3, sequenceid=24, filesize=5.0 K 2023-11-10 04:55:12,650 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/1974f7d317f64c329bae420ca51b136e as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/1974f7d317f64c329bae420ca51b136e 2023-11-10 04:55:12,654 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1974f7d317f64c329bae420ca51b136e 2023-11-10 04:55:12,654 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/1974f7d317f64c329bae420ca51b136e, entries=1, sequenceid=24, filesize=5.0 K 2023-11-10 04:55:12,655 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for ee1662c8ccd6e1d2468c8603ba66a3a6 in 1684ms, sequenceid=24, compaction requested=true 2023-11-10 04:55:12,655 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:12,655 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:12,655 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:12,655 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:12,655 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,655 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-10 04:55:12,655 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-10 04:55:12,657 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,657 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:12,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:12,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:12,657 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:12,657 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:12,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:12,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:12,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:12,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:12,659 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-10 04:55:12,659 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6de72467[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:12,659 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-10 04:55:12,659 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:12,660 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:12,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:12,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:12,660 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-10 04:55:12,660 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-10 04:55:12,660 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,660 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:12,660 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:12,667 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-10 04:55:12,667 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,667 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:12,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:12,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:12,667 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-10 04:55:12,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:12,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:12,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:12,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:12,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:12,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:12,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-10 04:55:12,670 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-10 04:55:12,670 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:12,670 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:12,670 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-10 04:55:12,670 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:12,672 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:12,672 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:12,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:12,672 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:12,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:12,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:12,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:12,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:12,673 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:12,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:12,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:12,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:12,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:12,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:12,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:12,674 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:12,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-10 04:55:12,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,676 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,676 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:12,676 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-10 04:55:12,676 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:12,676 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:12,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:12,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:12,676 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:12,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-10 04:55:12,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:12,676 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:12,676 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-10 04:55:12,677 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:12,677 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-10 04:55:12,677 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-10 04:55:12,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:12,677 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:12,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:12,677 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:12,777 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:12,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:12,779 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:12,779 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): 428db5f998f281799ebc91a16e0c51d8/family1 is initiating major compaction (all files) 2023-11-10 04:55:12,779 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:12,779 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:12,779 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of 428db5f998f281799ebc91a16e0c51d8/family1 in testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:12,780 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/8a93989bdfc14dea990dbac25df2ee02, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/c6696ef25fb345cbbf0b414f8500fedf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/54d22b6b43374e0ab599ff9a0a26dabc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0563518c4e2d4880b8cc81f80d202363, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/5fd5e1738faa40968a01401ae1384146] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp, totalSize=26.7 K 2023-11-10 04:55:12,780 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 8a93989bdfc14dea990dbac25df2ee02, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102317 2023-11-10 04:55:12,781 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting c6696ef25fb345cbbf0b414f8500fedf, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103722 2023-11-10 04:55:12,781 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 54d22b6b43374e0ab599ff9a0a26dabc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:12,782 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 0563518c4e2d4880b8cc81f80d202363, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:12,782 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 5fd5e1738faa40968a01401ae1384146, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-10 04:55:12,810 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): 428db5f998f281799ebc91a16e0c51d8#family1#compaction#107 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:13,221 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311107d9141a00d194cfebbf9a8857a18a3ff_del 2023-11-10 04:55:13,221 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311107d9141a00d194cfebbf9a8857a18a3ff_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311107d9141a00d194cfebbf9a8857a18a3ff_del 2023-11-10 04:55:13,638 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a958e426cc304ccc9e08bb63ad98c944 2023-11-10 04:55:13,639 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/a958e426cc304ccc9e08bb63ad98c944 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/a958e426cc304ccc9e08bb63ad98c944 2023-11-10 04:55:13,643 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a958e426cc304ccc9e08bb63ad98c944 2023-11-10 04:55:13,644 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 428db5f998f281799ebc91a16e0c51d8/family1 of 428db5f998f281799ebc91a16e0c51d8 into a958e426cc304ccc9e08bb63ad98c944(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:55:13,644 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:13,644 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:13,644 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): 428db5f998f281799ebc91a16e0c51d8/family2 is initiating major compaction (all files) 2023-11-10 04:55:13,644 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:13,644 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:13,644 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of 428db5f998f281799ebc91a16e0c51d8/family2 in testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:13,644 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ae4b2c41494145ffab0e5a5e9c571e89, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ad709bef78f04200a7e5a19bfc072ba1, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4ac76495f51f4d65856ae576d60f9971, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/a20efa896262435186ccb4afda7dcaa0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/dead17dad2744895bec8fa605f793efe] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp, totalSize=25.4 K 2023-11-10 04:55:13,645 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting ae4b2c41494145ffab0e5a5e9c571e89, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102317 2023-11-10 04:55:13,645 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting ad709bef78f04200a7e5a19bfc072ba1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103722 2023-11-10 04:55:13,645 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 4ac76495f51f4d65856ae576d60f9971, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:13,646 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting a20efa896262435186ccb4afda7dcaa0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:13,646 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting dead17dad2744895bec8fa605f793efe, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-10 04:55:13,658 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): 428db5f998f281799ebc91a16e0c51d8#family2#compaction#108 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:13,672 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231110ac20775a31204191a752802546493825_del 2023-11-10 04:55:13,673 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110ac20775a31204191a752802546493825_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110ac20775a31204191a752802546493825_del 2023-11-10 04:55:13,680 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/8a93989bdfc14dea990dbac25df2ee02, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/c6696ef25fb345cbbf0b414f8500fedf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/54d22b6b43374e0ab599ff9a0a26dabc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0563518c4e2d4880b8cc81f80d202363, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/5fd5e1738faa40968a01401ae1384146] to archive 2023-11-10 04:55:13,680 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:13,685 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/8a93989bdfc14dea990dbac25df2ee02 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/8a93989bdfc14dea990dbac25df2ee02 2023-11-10 04:55:13,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/c6696ef25fb345cbbf0b414f8500fedf to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/c6696ef25fb345cbbf0b414f8500fedf 2023-11-10 04:55:13,691 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/54d22b6b43374e0ab599ff9a0a26dabc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/54d22b6b43374e0ab599ff9a0a26dabc 2023-11-10 04:55:13,692 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0563518c4e2d4880b8cc81f80d202363 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0563518c4e2d4880b8cc81f80d202363 2023-11-10 04:55:13,693 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/5fd5e1738faa40968a01401ae1384146 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/5fd5e1738faa40968a01401ae1384146 2023-11-10 04:55:14,494 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b624e35fb764631826a9cba38968613 2023-11-10 04:55:14,494 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/4b624e35fb764631826a9cba38968613 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4b624e35fb764631826a9cba38968613 2023-11-10 04:55:14,498 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b624e35fb764631826a9cba38968613 2023-11-10 04:55:14,499 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 428db5f998f281799ebc91a16e0c51d8/family2 of 428db5f998f281799ebc91a16e0c51d8 into 4b624e35fb764631826a9cba38968613(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:55:14,499 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:14,499 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:14,499 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): 247092fa363b36c6fef420107a683723/family1 is initiating major compaction (all files) 2023-11-10 04:55:14,499 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:14,499 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:14,499 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of 247092fa363b36c6fef420107a683723/family1 in testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:14,499 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/343bb832e9ac4876816536ca263980ac, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/f76926bedfc44a90aae90e75491d98dd, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/b2d95126b7b4496c920348e7cc43b2be, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/18fa4d39f76c454b84c3556bc005dec3, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/7e24d760b113484bafd056ef97e2e2ab] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp, totalSize=26.7 K 2023-11-10 04:55:14,500 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 343bb832e9ac4876816536ca263980ac, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102316 2023-11-10 04:55:14,500 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting f76926bedfc44a90aae90e75491d98dd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103721 2023-11-10 04:55:14,501 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting b2d95126b7b4496c920348e7cc43b2be, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:14,501 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 18fa4d39f76c454b84c3556bc005dec3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:14,501 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 7e24d760b113484bafd056ef97e2e2ab, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-10 04:55:14,511 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): 247092fa363b36c6fef420107a683723#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:14,576 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ae4b2c41494145ffab0e5a5e9c571e89, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ad709bef78f04200a7e5a19bfc072ba1, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4ac76495f51f4d65856ae576d60f9971, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/a20efa896262435186ccb4afda7dcaa0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/dead17dad2744895bec8fa605f793efe] to archive 2023-11-10 04:55:14,577 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:14,579 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ae4b2c41494145ffab0e5a5e9c571e89 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ae4b2c41494145ffab0e5a5e9c571e89 2023-11-10 04:55:14,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ad709bef78f04200a7e5a19bfc072ba1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/ad709bef78f04200a7e5a19bfc072ba1 2023-11-10 04:55:14,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4ac76495f51f4d65856ae576d60f9971 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4ac76495f51f4d65856ae576d60f9971 2023-11-10 04:55:14,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/a20efa896262435186ccb4afda7dcaa0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/a20efa896262435186ccb4afda7dcaa0 2023-11-10 04:55:14,583 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/dead17dad2744895bec8fa605f793efe to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/dead17dad2744895bec8fa605f793efe 2023-11-10 04:55:14,922 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231110fd5c27ecba984c0d9e44475a814b0e71_del 2023-11-10 04:55:14,922 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110fd5c27ecba984c0d9e44475a814b0e71_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110fd5c27ecba984c0d9e44475a814b0e71_del 2023-11-10 04:55:15,736 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 37475d84e5d44d6d9d95593bdff221d2 2023-11-10 04:55:15,736 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/37475d84e5d44d6d9d95593bdff221d2 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/37475d84e5d44d6d9d95593bdff221d2 2023-11-10 04:55:15,740 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 37475d84e5d44d6d9d95593bdff221d2 2023-11-10 04:55:15,741 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 247092fa363b36c6fef420107a683723/family1 of 247092fa363b36c6fef420107a683723 into 37475d84e5d44d6d9d95593bdff221d2(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:15,741 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:15,741 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:15,741 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): 247092fa363b36c6fef420107a683723/family2 is initiating major compaction (all files) 2023-11-10 04:55:15,741 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:15,741 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:15,741 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of 247092fa363b36c6fef420107a683723/family2 in testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:15,742 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/097cfe0f69164ac28a93a645edd40a1e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/135a9c88def64bac9e94d078b28edbbf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/230ea277da404cf39dc02f7f034f7fd7, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c07b5707fdaa439788a90b8039e6f558, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c8f06fdd459341d7bd5b8ee8427685f9] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp, totalSize=25.4 K 2023-11-10 04:55:15,742 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 097cfe0f69164ac28a93a645edd40a1e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102316 2023-11-10 04:55:15,742 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 135a9c88def64bac9e94d078b28edbbf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103721 2023-11-10 04:55:15,743 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 230ea277da404cf39dc02f7f034f7fd7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:15,743 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting c07b5707fdaa439788a90b8039e6f558, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:15,743 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting c8f06fdd459341d7bd5b8ee8427685f9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-10 04:55:15,753 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): 247092fa363b36c6fef420107a683723#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:15,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/343bb832e9ac4876816536ca263980ac, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/f76926bedfc44a90aae90e75491d98dd, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/b2d95126b7b4496c920348e7cc43b2be, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/18fa4d39f76c454b84c3556bc005dec3, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/7e24d760b113484bafd056ef97e2e2ab] to archive 2023-11-10 04:55:15,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:15,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/343bb832e9ac4876816536ca263980ac to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/343bb832e9ac4876816536ca263980ac 2023-11-10 04:55:15,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/f76926bedfc44a90aae90e75491d98dd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/f76926bedfc44a90aae90e75491d98dd 2023-11-10 04:55:15,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/b2d95126b7b4496c920348e7cc43b2be to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/b2d95126b7b4496c920348e7cc43b2be 2023-11-10 04:55:15,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/18fa4d39f76c454b84c3556bc005dec3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/18fa4d39f76c454b84c3556bc005dec3 2023-11-10 04:55:15,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/7e24d760b113484bafd056ef97e2e2ab to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/7e24d760b113484bafd056ef97e2e2ab 2023-11-10 04:55:16,162 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231110c9311eb339294601a81ae478dd225527_del 2023-11-10 04:55:16,162 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110c9311eb339294601a81ae478dd225527_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110c9311eb339294601a81ae478dd225527_del 2023-11-10 04:55:16,977 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3d74c2b8257c465bb632b9d3c934a7c2 2023-11-10 04:55:16,977 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/3d74c2b8257c465bb632b9d3c934a7c2 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/3d74c2b8257c465bb632b9d3c934a7c2 2023-11-10 04:55:16,981 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3d74c2b8257c465bb632b9d3c934a7c2 2023-11-10 04:55:16,982 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 247092fa363b36c6fef420107a683723/family2 of 247092fa363b36c6fef420107a683723 into 3d74c2b8257c465bb632b9d3c934a7c2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:16,982 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:16,982 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:16,982 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): ee1662c8ccd6e1d2468c8603ba66a3a6/family1 is initiating major compaction (all files) 2023-11-10 04:55:16,982 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:16,982 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:16,982 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of ee1662c8ccd6e1d2468c8603ba66a3a6/family1 in testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:16,982 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/335c1b58d71a4b0088492a1457fb1527, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/1efd2e6cde824fe8afa345aaa743ec20, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/9a7ec7a970b54c7d84a57d195c12f5d5, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/61ea0ea6261a4724adaccf6dc92aa8fa, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/5a8e17569fb14fbfaa1e22bacad1a2d5] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp, totalSize=26.7 K 2023-11-10 04:55:16,983 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 335c1b58d71a4b0088492a1457fb1527, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102318 2023-11-10 04:55:16,983 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 1efd2e6cde824fe8afa345aaa743ec20, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103722 2023-11-10 04:55:16,983 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 9a7ec7a970b54c7d84a57d195c12f5d5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:16,984 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 61ea0ea6261a4724adaccf6dc92aa8fa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:16,984 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 5a8e17569fb14fbfaa1e22bacad1a2d5, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-10 04:55:16,995 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): ee1662c8ccd6e1d2468c8603ba66a3a6#family1#compaction#111 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:17,076 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/097cfe0f69164ac28a93a645edd40a1e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/135a9c88def64bac9e94d078b28edbbf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/230ea277da404cf39dc02f7f034f7fd7, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c07b5707fdaa439788a90b8039e6f558, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c8f06fdd459341d7bd5b8ee8427685f9] to archive 2023-11-10 04:55:17,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:17,078 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/097cfe0f69164ac28a93a645edd40a1e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/097cfe0f69164ac28a93a645edd40a1e 2023-11-10 04:55:17,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/135a9c88def64bac9e94d078b28edbbf to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/135a9c88def64bac9e94d078b28edbbf 2023-11-10 04:55:17,080 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/230ea277da404cf39dc02f7f034f7fd7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/230ea277da404cf39dc02f7f034f7fd7 2023-11-10 04:55:17,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c07b5707fdaa439788a90b8039e6f558 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c07b5707fdaa439788a90b8039e6f558 2023-11-10 04:55:17,082 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c8f06fdd459341d7bd5b8ee8427685f9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/c8f06fdd459341d7bd5b8ee8427685f9 2023-11-10 04:55:17,403 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023111030ed7a9ceb674a44bf2a2bf9fbfa682f_del 2023-11-10 04:55:17,403 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111030ed7a9ceb674a44bf2a2bf9fbfa682f_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111030ed7a9ceb674a44bf2a2bf9fbfa682f_del 2023-11-10 04:55:18,217 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25c85cec0631428c9ceff71da828a39c 2023-11-10 04:55:18,218 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/25c85cec0631428c9ceff71da828a39c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/25c85cec0631428c9ceff71da828a39c 2023-11-10 04:55:18,222 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25c85cec0631428c9ceff71da828a39c 2023-11-10 04:55:18,223 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ee1662c8ccd6e1d2468c8603ba66a3a6/family1 of ee1662c8ccd6e1d2468c8603ba66a3a6 into 25c85cec0631428c9ceff71da828a39c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:18,223 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:18,223 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:18,223 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): ee1662c8ccd6e1d2468c8603ba66a3a6/family2 is initiating major compaction (all files) 2023-11-10 04:55:18,223 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:18,223 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:18,223 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of ee1662c8ccd6e1d2468c8603ba66a3a6/family2 in testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:18,224 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/fdaea769dff24e2f87ae03469c289c42, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/a63356bdf54c4d7b9c03c16621133a28, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/3dfe247256a54b92a5549e8aa06c7a79, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/1974f7d317f64c329bae420ca51b136e] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp, totalSize=25.4 K 2023-11-10 04:55:18,224 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting fdaea769dff24e2f87ae03469c289c42, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102318 2023-11-10 04:55:18,224 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 8dd99ea06da34d1aa6b90f0c0cd05ed8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103722 2023-11-10 04:55:18,225 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting a63356bdf54c4d7b9c03c16621133a28, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:18,225 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 3dfe247256a54b92a5549e8aa06c7a79, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:18,225 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 1974f7d317f64c329bae420ca51b136e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-10 04:55:18,240 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): ee1662c8ccd6e1d2468c8603ba66a3a6#family2#compaction#112 average throughput is 0.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:18,272 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/335c1b58d71a4b0088492a1457fb1527, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/1efd2e6cde824fe8afa345aaa743ec20, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/9a7ec7a970b54c7d84a57d195c12f5d5, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/61ea0ea6261a4724adaccf6dc92aa8fa, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/5a8e17569fb14fbfaa1e22bacad1a2d5] to archive 2023-11-10 04:55:18,273 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:18,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/335c1b58d71a4b0088492a1457fb1527 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/335c1b58d71a4b0088492a1457fb1527 2023-11-10 04:55:18,275 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/1efd2e6cde824fe8afa345aaa743ec20 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/1efd2e6cde824fe8afa345aaa743ec20 2023-11-10 04:55:18,276 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/9a7ec7a970b54c7d84a57d195c12f5d5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/9a7ec7a970b54c7d84a57d195c12f5d5 2023-11-10 04:55:18,277 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/61ea0ea6261a4724adaccf6dc92aa8fa to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/61ea0ea6261a4724adaccf6dc92aa8fa 2023-11-10 04:55:18,278 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/5a8e17569fb14fbfaa1e22bacad1a2d5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/5a8e17569fb14fbfaa1e22bacad1a2d5 2023-11-10 04:55:18,648 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023111026590343a8d9400380a3b90a765672e0_del 2023-11-10 04:55:18,648 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111026590343a8d9400380a3b90a765672e0_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111026590343a8d9400380a3b90a765672e0_del 2023-11-10 04:55:19,470 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0bc19922f59449cadf3044dc69ce060 2023-11-10 04:55:19,471 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/f0bc19922f59449cadf3044dc69ce060 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/f0bc19922f59449cadf3044dc69ce060 2023-11-10 04:55:19,475 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0bc19922f59449cadf3044dc69ce060 2023-11-10 04:55:19,476 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ee1662c8ccd6e1d2468c8603ba66a3a6/family2 of ee1662c8ccd6e1d2468c8603ba66a3a6 into f0bc19922f59449cadf3044dc69ce060(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:19,476 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:19,481 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:19,485 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:19,489 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:19,490 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:19,497 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-10 04:55:19,541 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-11-10 04:55:19,545 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231110fd5c27ecba984c0d9e44475a814b0e71_del 2023-11-10 04:55:19,548 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311107d9141a00d194cfebbf9a8857a18a3ff_del 2023-11-10 04:55:19,552 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023111030ed7a9ceb674a44bf2a2bf9fbfa682f_del 2023-11-10 04:55:19,552 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:19,554 DEBUG [Listener at localhost/35765] zookeeper.ReadOnlyZKClient(139): Connect 0x36829195 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:55:19,559 DEBUG [Listener at localhost/35765] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@68649dab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:55:19,559 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:19,560 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:19,560 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:19,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/fdaea769dff24e2f87ae03469c289c42, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/a63356bdf54c4d7b9c03c16621133a28, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/3dfe247256a54b92a5549e8aa06c7a79, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/1974f7d317f64c329bae420ca51b136e] to archive 2023-11-10 04:55:19,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:19,579 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/fdaea769dff24e2f87ae03469c289c42 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/fdaea769dff24e2f87ae03469c289c42 2023-11-10 04:55:19,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/8dd99ea06da34d1aa6b90f0c0cd05ed8 2023-11-10 04:55:19,583 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/a63356bdf54c4d7b9c03c16621133a28 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/a63356bdf54c4d7b9c03c16621133a28 2023-11-10 04:55:19,586 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/3dfe247256a54b92a5549e8aa06c7a79 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/3dfe247256a54b92a5549e8aa06c7a79 2023-11-10 04:55:19,587 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/1974f7d317f64c329bae420ca51b136e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/1974f7d317f64c329bae420ca51b136e 2023-11-10 04:55:20,098 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023111033934548ef404fcbbb3bb2fcd457dabc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023111033934548ef404fcbbb3bb2fcd457dabc 2023-11-10 04:55:20,099 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:20,116 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:55:20,120 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37498, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:55:20,125 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311104fd09bdfd54a4e248077d0b995ea816a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311104fd09bdfd54a4e248077d0b995ea816a 2023-11-10 04:55:20,126 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:20,128 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:55:20,130 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37462, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:55:20,148 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110/family1/451d4e61ae7f42a5ac8f21990586790e first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-11-10 04:55:20,149 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110/family1/451d4e61ae7f42a5ac8f21990586790e}] 2023-11-10 04:55:20,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110/family1/451d4e61ae7f42a5ac8f21990586790e for inclusion in 247092fa363b36c6fef420107a683723/family1 2023-11-10 04:55:20,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-11-10 04:55:20,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last=1 2023-11-10 04:55:20,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:20,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110/family1/451d4e61ae7f42a5ac8f21990586790e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2ldn4tngec643935takoi5qr3ujvsaajfb6d2116cmgpdnt7kln4dvf2spp160t2/family1/451d4e61ae7f42a5ac8f21990586790e 2023-11-10 04:55:20,164 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110/family1/b98ca2f1da4c40dd877e3c4f3fe3dab9 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-11-10 04:55:20,166 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110/family1/b98ca2f1da4c40dd877e3c4f3fe3dab9}] 2023-11-10 04:55:20,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2ldn4tngec643935takoi5qr3ujvsaajfb6d2116cmgpdnt7kln4dvf2spp160t2/family1/451d4e61ae7f42a5ac8f21990586790e as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/c652d0294d2c41a28ab11b1ba95e72d2_SeqId_29_ 2023-11-10 04:55:20,168 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2ldn4tngec643935takoi5qr3ujvsaajfb6d2116cmgpdnt7kln4dvf2spp160t2/family1/451d4e61ae7f42a5ac8f21990586790e into 247092fa363b36c6fef420107a683723/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/c652d0294d2c41a28ab11b1ba95e72d2_SeqId_29_ - updating store file list. 2023-11-10 04:55:20,168 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110/family1/b98ca2f1da4c40dd877e3c4f3fe3dab9 for inclusion in 428db5f998f281799ebc91a16e0c51d8/family1 2023-11-10 04:55:20,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-11-10 04:55:20,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-10 04:55:20,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:20,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110/family1/b98ca2f1da4c40dd877e3c4f3fe3dab9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__u2io9o9c2vubravg471s6j4vigtbkvg9ird6vihfsbfnfgev597aji0vkrvtctld/family1/b98ca2f1da4c40dd877e3c4f3fe3dab9 2023-11-10 04:55:20,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/c652d0294d2c41a28ab11b1ba95e72d2_SeqId_29_ into 247092fa363b36c6fef420107a683723/family1 2023-11-10 04:55:20,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2ldn4tngec643935takoi5qr3ujvsaajfb6d2116cmgpdnt7kln4dvf2spp160t2/family1/451d4e61ae7f42a5ac8f21990586790e into 247092fa363b36c6fef420107a683723/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/c652d0294d2c41a28ab11b1ba95e72d2_SeqId_29_) 2023-11-10 04:55:20,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2ldn4tngec643935takoi5qr3ujvsaajfb6d2116cmgpdnt7kln4dvf2spp160t2/family1/451d4e61ae7f42a5ac8f21990586790e 2023-11-10 04:55:20,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__u2io9o9c2vubravg471s6j4vigtbkvg9ird6vihfsbfnfgev597aji0vkrvtctld/family1/b98ca2f1da4c40dd877e3c4f3fe3dab9 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/facba5f4dc9149a78e4233ce62f49610_SeqId_29_ 2023-11-10 04:55:20,174 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:20,175 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__u2io9o9c2vubravg471s6j4vigtbkvg9ird6vihfsbfnfgev597aji0vkrvtctld/family1/b98ca2f1da4c40dd877e3c4f3fe3dab9 into 428db5f998f281799ebc91a16e0c51d8/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/facba5f4dc9149a78e4233ce62f49610_SeqId_29_ - updating store file list. 2023-11-10 04:55:20,175 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:20,177 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311106b48e94e65554c4893ee5e1202c8783c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311106b48e94e65554c4893ee5e1202c8783c 2023-11-10 04:55:20,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/facba5f4dc9149a78e4233ce62f49610_SeqId_29_ into 428db5f998f281799ebc91a16e0c51d8/family1 2023-11-10 04:55:20,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__u2io9o9c2vubravg471s6j4vigtbkvg9ird6vihfsbfnfgev597aji0vkrvtctld/family1/b98ca2f1da4c40dd877e3c4f3fe3dab9 into 428db5f998f281799ebc91a16e0c51d8/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/facba5f4dc9149a78e4233ce62f49610_SeqId_29_) 2023-11-10 04:55:20,179 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110778980a6a03548eaa0eb0261594b0c47 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110778980a6a03548eaa0eb0261594b0c47 2023-11-10 04:55:20,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__u2io9o9c2vubravg471s6j4vigtbkvg9ird6vihfsbfnfgev597aji0vkrvtctld/family1/b98ca2f1da4c40dd877e3c4f3fe3dab9 2023-11-10 04:55:20,180 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311108a9f8f2e160e4ae297ff5d6758f35d78 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311108a9f8f2e160e4ae297ff5d6758f35d78 2023-11-10 04:55:20,181 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:20,185 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:20,185 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110efc6c47081fd44c791c6b6fe97a0558f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110efc6c47081fd44c791c6b6fe97a0558f 2023-11-10 04:55:20,185 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-10 04:55:20,185 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110' 2023-11-10 04:55:20,186 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:20,186 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110aa20f878baf844b0bdeca7e8694461f4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110aa20f878baf844b0bdeca7e8694461f4 2023-11-10 04:55:20,188 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110b0d110932bae45359e8a228021cd70f1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110b0d110932bae45359e8a228021cd70f1 2023-11-10 04:55:20,189 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110d2c8fc62e00a4510a1d823cffe6d60a4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110d2c8fc62e00a4510a1d823cffe6d60a4 2023-11-10 04:55:20,191 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110ed18e5da3f2348169787daf780e530cd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231110ed18e5da3f2348169787daf780e530cd 2023-11-10 04:55:20,191 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-10 04:55:20,191 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110' 2023-11-10 04:55:20,191 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:20,511 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110c182e230c8f648fd8db79a5855937854 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110c182e230c8f648fd8db79a5855937854 2023-11-10 04:55:20,512 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:20,541 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110/family1/2ed85cff9a4c449cbc7ee19e42f68ed2 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-11-10 04:55:20,542 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110/family1/2ed85cff9a4c449cbc7ee19e42f68ed2}] 2023-11-10 04:55:20,547 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110/family1/2ed85cff9a4c449cbc7ee19e42f68ed2 for inclusion in ee1662c8ccd6e1d2468c8603ba66a3a6/family1 2023-11-10 04:55:20,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-11-10 04:55:20,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-10 04:55:20,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:20,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110/family1/2ed85cff9a4c449cbc7ee19e42f68ed2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__jnegu7gnbrngsds79tcb1dkis6cu6fl6r93196pfund3jpr6a9nvvel3oiesbnba/family1/2ed85cff9a4c449cbc7ee19e42f68ed2 2023-11-10 04:55:20,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__jnegu7gnbrngsds79tcb1dkis6cu6fl6r93196pfund3jpr6a9nvvel3oiesbnba/family1/2ed85cff9a4c449cbc7ee19e42f68ed2 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/e6b2b83d3512400aab2d96c521098020_SeqId_29_ 2023-11-10 04:55:20,556 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__jnegu7gnbrngsds79tcb1dkis6cu6fl6r93196pfund3jpr6a9nvvel3oiesbnba/family1/2ed85cff9a4c449cbc7ee19e42f68ed2 into ee1662c8ccd6e1d2468c8603ba66a3a6/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/e6b2b83d3512400aab2d96c521098020_SeqId_29_ - updating store file list. 2023-11-10 04:55:20,560 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/e6b2b83d3512400aab2d96c521098020_SeqId_29_ into ee1662c8ccd6e1d2468c8603ba66a3a6/family1 2023-11-10 04:55:20,560 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__jnegu7gnbrngsds79tcb1dkis6cu6fl6r93196pfund3jpr6a9nvvel3oiesbnba/family1/2ed85cff9a4c449cbc7ee19e42f68ed2 into ee1662c8ccd6e1d2468c8603ba66a3a6/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/e6b2b83d3512400aab2d96c521098020_SeqId_29_) 2023-11-10 04:55:20,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__jnegu7gnbrngsds79tcb1dkis6cu6fl6r93196pfund3jpr6a9nvvel3oiesbnba/family1/2ed85cff9a4c449cbc7ee19e42f68ed2 2023-11-10 04:55:20,563 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:20,564 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:20,565 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111000281f5ebd834e8f9eb29aa07846bab0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111000281f5ebd834e8f9eb29aa07846bab0 2023-11-10 04:55:20,566 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110a49554fc563041f8a8a1665675a6885b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110a49554fc563041f8a8a1665675a6885b 2023-11-10 04:55:20,567 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110b0ceb0985d4447449efb0ad15104d7fd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110b0ceb0985d4447449efb0ad15104d7fd 2023-11-10 04:55:20,568 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110cefd0f548db540a78c386a0d2a56fde4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110cefd0f548db540a78c386a0d2a56fde4 2023-11-10 04:55:20,568 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-10 04:55:20,568 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110' 2023-11-10 04:55:20,569 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:20,569 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:20,569 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:20,569 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110fd5c27ecba984c0d9e44475a814b0e71_del] 2023-11-10 04:55:20,570 DEBUG [Listener at localhost/35765] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:20,571 DEBUG [Listener at localhost/35765] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110fd5c27ecba984c0d9e44475a814b0e71_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110fd5c27ecba984c0d9e44475a814b0e71_del 2023-11-10 04:55:20,571 INFO [Listener at localhost/35765] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-10 04:55:20,571 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311107d9141a00d194cfebbf9a8857a18a3ff_del] 2023-11-10 04:55:20,572 DEBUG [Listener at localhost/35765] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:20,573 DEBUG [Listener at localhost/35765] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311107d9141a00d194cfebbf9a8857a18a3ff_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311107d9141a00d194cfebbf9a8857a18a3ff_del 2023-11-10 04:55:20,573 INFO [Listener at localhost/35765] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-10 04:55:20,573 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111030ed7a9ceb674a44bf2a2bf9fbfa682f_del] 2023-11-10 04:55:20,574 DEBUG [Listener at localhost/35765] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:20,575 DEBUG [Listener at localhost/35765] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111030ed7a9ceb674a44bf2a2bf9fbfa682f_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111030ed7a9ceb674a44bf2a2bf9fbfa682f_del 2023-11-10 04:55:20,575 INFO [Listener at localhost/35765] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-10 04:55:20,581 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:20,583 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:20,590 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:20,591 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:20,598 INFO [Listener at localhost/35765] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-11-10 04:55:20,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-11-10 04:55:20,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-11-10 04:55:20,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-10 04:55:20,601 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592120601"}]},"ts":"1699592120601"} 2023-11-10 04:55:20,603 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-11-10 04:55:20,605 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-11-10 04:55:20,606 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, UNASSIGN}] 2023-11-10 04:55:20,607 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, UNASSIGN 2023-11-10 04:55:20,607 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, UNASSIGN 2023-11-10 04:55:20,607 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, UNASSIGN 2023-11-10 04:55:20,608 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=247092fa363b36c6fef420107a683723, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:20,608 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592120608"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592120608"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592120608"}]},"ts":"1699592120608"} 2023-11-10 04:55:20,609 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=428db5f998f281799ebc91a16e0c51d8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:20,609 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592120608"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592120608"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592120608"}]},"ts":"1699592120608"} 2023-11-10 04:55:20,609 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=ee1662c8ccd6e1d2468c8603ba66a3a6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:20,609 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592120609"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592120609"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592120609"}]},"ts":"1699592120609"} 2023-11-10 04:55:20,611 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure 428db5f998f281799ebc91a16e0c51d8, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:20,611 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure 247092fa363b36c6fef420107a683723, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:20,613 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure ee1662c8ccd6e1d2468c8603ba66a3a6, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:20,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-10 04:55:20,764 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:20,764 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 247092fa363b36c6fef420107a683723 2023-11-10 04:55:20,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 247092fa363b36c6fef420107a683723, disabling compactions & flushes 2023-11-10 04:55:20,764 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:20,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:20,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. after waiting 0 ms 2023-11-10 04:55:20,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:20,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-10 04:55:20,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:20,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:20,774 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=247092fa363b36c6fef420107a683723, regionState=CLOSED 2023-11-10 04:55:20,775 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592120774"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592120774"}]},"ts":"1699592120774"} 2023-11-10 04:55:20,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 247092fa363b36c6fef420107a683723 2023-11-10 04:55:20,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:20,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 428db5f998f281799ebc91a16e0c51d8, disabling compactions & flushes 2023-11-10 04:55:20,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:20,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:20,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. after waiting 0 ms 2023-11-10 04:55:20,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:20,779 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=93, resume processing ppid=89 2023-11-10 04:55:20,779 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure 247092fa363b36c6fef420107a683723, server=jenkins-hbase4.apache.org,36419,1699592002485 in 166 msec 2023-11-10 04:55:20,780 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, UNASSIGN in 173 msec 2023-11-10 04:55:20,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-10 04:55:20,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:20,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:20,785 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=428db5f998f281799ebc91a16e0c51d8, regionState=CLOSED 2023-11-10 04:55:20,785 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592120785"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592120785"}]},"ts":"1699592120785"} 2023-11-10 04:55:20,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:20,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:20,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ee1662c8ccd6e1d2468c8603ba66a3a6, disabling compactions & flushes 2023-11-10 04:55:20,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:20,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:20,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. after waiting 0 ms 2023-11-10 04:55:20,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:20,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=92, resume processing ppid=90 2023-11-10 04:55:20,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure 428db5f998f281799ebc91a16e0c51d8, server=jenkins-hbase4.apache.org,36419,1699592002485 in 175 msec 2023-11-10 04:55:20,790 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, UNASSIGN in 182 msec 2023-11-10 04:55:20,799 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-10 04:55:20,800 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:20,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:20,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:20,802 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=ee1662c8ccd6e1d2468c8603ba66a3a6, regionState=CLOSED 2023-11-10 04:55:20,802 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592120801"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592120801"}]},"ts":"1699592120801"} 2023-11-10 04:55:20,804 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=94, resume processing ppid=91 2023-11-10 04:55:20,804 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure ee1662c8ccd6e1d2468c8603ba66a3a6, server=jenkins-hbase4.apache.org,36419,1699592002485 in 190 msec 2023-11-10 04:55:20,806 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=91, resume processing ppid=88 2023-11-10 04:55:20,806 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, UNASSIGN in 198 msec 2023-11-10 04:55:20,806 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592120806"}]},"ts":"1699592120806"} 2023-11-10 04:55:20,807 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-11-10 04:55:20,809 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-11-10 04:55:20,810 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 211 msec 2023-11-10 04:55:20,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-10 04:55:20,904 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-11-10 04:55:20,917 INFO [Listener at localhost/35765] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1699592108764-1699592120917 2023-11-10 04:55:20,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-10 04:55:20,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1699592120918 (current time:1699592120918). 2023-11-10 04:55:20,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-11-10 04:55:20,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] zookeeper.ReadOnlyZKClient(139): Connect 0x6700d886 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:55:20,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@54a7b5ed, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:55:20,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:55:20,930 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37512, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:55:20,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6700d886 to 127.0.0.1:51169 2023-11-10 04:55:20,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:55:20,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-11-10 04:55:20,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(695): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-10 04:55:20,934 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot, current working dir path: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1699592108764-1699592120917 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotDisabledTable(SnapshotManager.java:541) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:698) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-11-10 04:55:20,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-10 04:55:20,936 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-10 04:55:20,937 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1699592720937 2023-11-10 04:55:20,937 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1699592108764-1699592120917 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-11-10 04:55:20,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(700): Started snapshot: { ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-10 04:55:20,937 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:20,938 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-11-10 04:55:21,038 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:21,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:21,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-10 04:55:21,039 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-11-10 04:55:21,239 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:21,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:21,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-10 04:55:21,241 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-11-10 04:55:21,349 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-11-10 04:55:21,351 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-10 04:55:21,351 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-10 04:55:21,351 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-10 04:55:21,351 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-10 04:55:21,351 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-10 04:55:21,351 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-10 04:55:21,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-10 04:55:21,352 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-10 04:55:21,360 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/25c85cec0631428c9ceff71da828a39c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/e6b2b83d3512400aab2d96c521098020_SeqId_29_] hfiles 2023-11-10 04:55:21,360 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/25c85cec0631428c9ceff71da828a39c 2023-11-10 04:55:21,360 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/e6b2b83d3512400aab2d96c521098020_SeqId_29_ 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311104fd09bdfd54a4e248077d0b995ea816a, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110c182e230c8f648fd8db79a5855937854, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023111033934548ef404fcbbb3bb2fcd457dabc] hfiles 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/37475d84e5d44d6d9d95593bdff221d2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/c652d0294d2c41a28ab11b1ba95e72d2_SeqId_29_] hfiles 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/37475d84e5d44d6d9d95593bdff221d2 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311104fd09bdfd54a4e248077d0b995ea816a 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/c652d0294d2c41a28ab11b1ba95e72d2_SeqId_29_ 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/f0bc19922f59449cadf3044dc69ce060] hfiles 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110c182e230c8f648fd8db79a5855937854 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/f0bc19922f59449cadf3044dc69ce060 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023111033934548ef404fcbbb3bb2fcd457dabc 2023-11-10 04:55:21,362 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/a958e426cc304ccc9e08bb63ad98c944, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/facba5f4dc9149a78e4233ce62f49610_SeqId_29_] hfiles 2023-11-10 04:55:21,363 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/a958e426cc304ccc9e08bb63ad98c944 2023-11-10 04:55:21,363 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/facba5f4dc9149a78e4233ce62f49610_SeqId_29_ 2023-11-10 04:55:21,363 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/3d74c2b8257c465bb632b9d3c934a7c2] hfiles 2023-11-10 04:55:21,364 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/3d74c2b8257c465bb632b9d3c934a7c2 2023-11-10 04:55:21,364 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4b624e35fb764631826a9cba38968613] hfiles 2023-11-10 04:55:21,364 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4b624e35fb764631826a9cba38968613 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110383e4615c7b74a309ccb47b415522473, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110682a56e6bb8f493581f91f5b1006c728, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110a68d06cb679f4f328510a8bc60678329, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110ac20775a31204191a752802546493825_del, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110f5ff6c793ec14b4292dea798e6a487bd, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111003c9829c1ab445fbb1840b9e244a99dc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111026590343a8d9400380a3b90a765672e0_del, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110a7700e7ca441486fb327e27f060c222e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110aadf2e04b6f24e108e183dca05a37868, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110e8a8ae26e34548f698d5b7198923d293, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111033e0034756e34f0db93bbc10f36c3355, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111048cf145946e0415dbff474330468d120, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311109737724df754430a91cf28554ad175b8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110c9311eb339294601a81ae478dd225527_del, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110f8196a9cfd674d4391e4aac41410be17] hfiles 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110383e4615c7b74a309ccb47b415522473 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110682a56e6bb8f493581f91f5b1006c728 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110a68d06cb679f4f328510a8bc60678329 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110ac20775a31204191a752802546493825_del 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110f5ff6c793ec14b4292dea798e6a487bd 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111003c9829c1ab445fbb1840b9e244a99dc 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111026590343a8d9400380a3b90a765672e0_del 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110a7700e7ca441486fb327e27f060c222e 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110aadf2e04b6f24e108e183dca05a37868 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231110e8a8ae26e34548f698d5b7198923d293 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111033e0034756e34f0db93bbc10f36c3355 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023111048cf145946e0415dbff474330468d120 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202311109737724df754430a91cf28554ad175b8 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110c9311eb339294601a81ae478dd225527_del 2023-11-10 04:55:21,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110f8196a9cfd674d4391e4aac41410be17 2023-11-10 04:55:21,541 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:21,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:21,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-10 04:55:21,543 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-11-10 04:55:21,783 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-11-10 04:55:21,783 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1699592108764-1699592120917 2023-11-10 04:55:21,784 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1699592108764-1699592120917 2023-11-10 04:55:22,043 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:22,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:22,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-10 04:55:22,044 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-11-10 04:55:22,230 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1699592108764-1699592120917 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/hbase-failsafe-snaptb-1699592108764-1699592120917 2023-11-10 04:55:22,231 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1699592108764-1699592120917 of table testCompactionWithHFileLink completed 2023-11-10 04:55:22,231 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1699592108764-1699592120917 2023-11-10 04:55:22,231 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1699592108764-1699592120917 2023-11-10 04:55:22,231 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1699592108764-1699592120917 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1699592120937Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1699592121349 (+412 ms)Consolidate snapshot: hbase-failsafe-snaptb-1699592108764-1699592120917 at 1699592121783 (+434 ms)Loading Region manifests for hbase-failsafe-snaptb-1699592108764-1699592120917 at 1699592121783Writing data manifest for hbase-failsafe-snaptb-1699592108764-1699592120917 at 1699592121788 (+5 ms)Verifying snapshot: hbase-failsafe-snaptb-1699592108764-1699592120917 at 1699592122209 (+421 ms)Snapshot hbase-failsafe-snaptb-1699592108764-1699592120917 of table testCompactionWithHFileLink completed at 1699592122231 (+22 ms) 2023-11-10 04:55:22,233 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-10 04:55:22,234 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.2980 sec 2023-11-10 04:55:23,045 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-10 04:55:23,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-10 04:55:23,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(403): Snapshot '{ ss=hbase-failsafe-snaptb-1699592108764-1699592120917 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-11-10 04:55:23,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1699592108764" table: "testCompactionWithHFileLink" creation_time: 1699592108775 type: FLUSH version: 2 ttl: 0 ) 2023-11-10 04:55:23,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(916): Restore snapshot=snaptb-1699592108764 as table=testCompactionWithHFileLink 2023-11-10 04:55:23,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-10 04:55:23,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-10 04:55:23,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-10 04:55:23,503 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-11-10 04:55:23,504 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-11-10 04:55:23,504 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-11-10 04:55:23,505 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-10 04:55:23,511 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1699592108764" table: "testCompactionWithHFileLink" creation_time: 1699592108775 type: FLUSH version: 2 ttl: 0 2023-11-10 04:55:23,511 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink 2023-11-10 04:55:23,516 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-11-10 04:55:23,520 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311104fd09bdfd54a4e248077d0b995ea816a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311104fd09bdfd54a4e248077d0b995ea816a 2023-11-10 04:55:23,521 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110c182e230c8f648fd8db79a5855937854 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20231110c182e230c8f648fd8db79a5855937854 2023-11-10 04:55:23,522 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023111033934548ef404fcbbb3bb2fcd457dabc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023111033934548ef404fcbbb3bb2fcd457dabc 2023-11-10 04:55:23,522 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231110aa20f878baf844b0bdeca7e8694461f4 of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,525 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231110b0d110932bae45359e8a228021cd70f1 of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,528 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231110d2c8fc62e00a4510a1d823cffe6d60a4 of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,531 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231110ed18e5da3f2348169787daf780e530cd of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,534 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023111000281f5ebd834e8f9eb29aa07846bab0 of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,537 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20231110a49554fc563041f8a8a1665675a6885b of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,540 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20231110b0ceb0985d4447449efb0ad15104d7fd of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,543 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20231110cefd0f548db540a78c386a0d2a56fde4 of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,547 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202311106b48e94e65554c4893ee5e1202c8783c of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,550 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20231110778980a6a03548eaa0eb0261594b0c47 of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,553 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202311108a9f8f2e160e4ae297ff5d6758f35d78 of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,556 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20231110efc6c47081fd44c791c6b6fe97a0558f of snapshot=snaptb-1699592108764 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-10 04:55:23,560 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110c9311eb339294601a81ae478dd225527_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231110c9311eb339294601a81ae478dd225527_del 2023-11-10 04:55:23,561 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110ac20775a31204191a752802546493825_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231110ac20775a31204191a752802546493825_del 2023-11-10 04:55:23,562 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111026590343a8d9400380a3b90a765672e0_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023111026590343a8d9400380a3b90a765672e0_del 2023-11-10 04:55:23,562 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 247092fa363b36c6fef420107a683723 2023-11-10 04:55:23,562 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:23,563 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:23,566 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/37475d84e5d44d6d9d95593bdff221d2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/37475d84e5d44d6d9d95593bdff221d2 2023-11-10 04:55:23,566 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/25c85cec0631428c9ceff71da828a39c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/25c85cec0631428c9ceff71da828a39c 2023-11-10 04:55:23,566 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/a958e426cc304ccc9e08bb63ad98c944 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/a958e426cc304ccc9e08bb63ad98c944 2023-11-10 04:55:23,568 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/c652d0294d2c41a28ab11b1ba95e72d2_SeqId_29_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/c652d0294d2c41a28ab11b1ba95e72d2_SeqId_29_ 2023-11-10 04:55:23,568 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 343bb832e9ac4876816536ca263980ac of snapshot=snaptb-1699592108764 to region=247092fa363b36c6fef420107a683723 table=testCompactionWithHFileLink 2023-11-10 04:55:23,568 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/e6b2b83d3512400aab2d96c521098020_SeqId_29_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/e6b2b83d3512400aab2d96c521098020_SeqId_29_ 2023-11-10 04:55:23,568 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/facba5f4dc9149a78e4233ce62f49610_SeqId_29_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/facba5f4dc9149a78e4233ce62f49610_SeqId_29_ 2023-11-10 04:55:23,568 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 335c1b58d71a4b0088492a1457fb1527 of snapshot=snaptb-1699592108764 to region=ee1662c8ccd6e1d2468c8603ba66a3a6 table=testCompactionWithHFileLink 2023-11-10 04:55:23,569 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8a93989bdfc14dea990dbac25df2ee02 of snapshot=snaptb-1699592108764 to region=428db5f998f281799ebc91a16e0c51d8 table=testCompactionWithHFileLink 2023-11-10 04:55:23,573 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1efd2e6cde824fe8afa345aaa743ec20 of snapshot=snaptb-1699592108764 to region=ee1662c8ccd6e1d2468c8603ba66a3a6 table=testCompactionWithHFileLink 2023-11-10 04:55:23,573 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f76926bedfc44a90aae90e75491d98dd of snapshot=snaptb-1699592108764 to region=247092fa363b36c6fef420107a683723 table=testCompactionWithHFileLink 2023-11-10 04:55:23,573 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c6696ef25fb345cbbf0b414f8500fedf of snapshot=snaptb-1699592108764 to region=428db5f998f281799ebc91a16e0c51d8 table=testCompactionWithHFileLink 2023-11-10 04:55:23,577 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9a7ec7a970b54c7d84a57d195c12f5d5 of snapshot=snaptb-1699592108764 to region=ee1662c8ccd6e1d2468c8603ba66a3a6 table=testCompactionWithHFileLink 2023-11-10 04:55:23,577 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b2d95126b7b4496c920348e7cc43b2be of snapshot=snaptb-1699592108764 to region=247092fa363b36c6fef420107a683723 table=testCompactionWithHFileLink 2023-11-10 04:55:23,577 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 54d22b6b43374e0ab599ff9a0a26dabc of snapshot=snaptb-1699592108764 to region=428db5f998f281799ebc91a16e0c51d8 table=testCompactionWithHFileLink 2023-11-10 04:55:23,585 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 61ea0ea6261a4724adaccf6dc92aa8fa of snapshot=snaptb-1699592108764 to region=ee1662c8ccd6e1d2468c8603ba66a3a6 table=testCompactionWithHFileLink 2023-11-10 04:55:23,586 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 18fa4d39f76c454b84c3556bc005dec3 of snapshot=snaptb-1699592108764 to region=247092fa363b36c6fef420107a683723 table=testCompactionWithHFileLink 2023-11-10 04:55:23,586 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0563518c4e2d4880b8cc81f80d202363 of snapshot=snaptb-1699592108764 to region=428db5f998f281799ebc91a16e0c51d8 table=testCompactionWithHFileLink 2023-11-10 04:55:23,592 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/f0bc19922f59449cadf3044dc69ce060 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/f0bc19922f59449cadf3044dc69ce060 2023-11-10 04:55:23,592 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fdaea769dff24e2f87ae03469c289c42 of snapshot=snaptb-1699592108764 to region=ee1662c8ccd6e1d2468c8603ba66a3a6 table=testCompactionWithHFileLink 2023-11-10 04:55:23,592 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4b624e35fb764631826a9cba38968613 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/4b624e35fb764631826a9cba38968613 2023-11-10 04:55:23,592 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ae4b2c41494145ffab0e5a5e9c571e89 of snapshot=snaptb-1699592108764 to region=428db5f998f281799ebc91a16e0c51d8 table=testCompactionWithHFileLink 2023-11-10 04:55:23,592 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/3d74c2b8257c465bb632b9d3c934a7c2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/3d74c2b8257c465bb632b9d3c934a7c2 2023-11-10 04:55:23,592 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 097cfe0f69164ac28a93a645edd40a1e of snapshot=snaptb-1699592108764 to region=247092fa363b36c6fef420107a683723 table=testCompactionWithHFileLink 2023-11-10 04:55:23,595 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ad709bef78f04200a7e5a19bfc072ba1 of snapshot=snaptb-1699592108764 to region=428db5f998f281799ebc91a16e0c51d8 table=testCompactionWithHFileLink 2023-11-10 04:55:23,595 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 135a9c88def64bac9e94d078b28edbbf of snapshot=snaptb-1699592108764 to region=247092fa363b36c6fef420107a683723 table=testCompactionWithHFileLink 2023-11-10 04:55:23,596 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8dd99ea06da34d1aa6b90f0c0cd05ed8 of snapshot=snaptb-1699592108764 to region=ee1662c8ccd6e1d2468c8603ba66a3a6 table=testCompactionWithHFileLink 2023-11-10 04:55:23,599 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 230ea277da404cf39dc02f7f034f7fd7 of snapshot=snaptb-1699592108764 to region=247092fa363b36c6fef420107a683723 table=testCompactionWithHFileLink 2023-11-10 04:55:23,599 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4ac76495f51f4d65856ae576d60f9971 of snapshot=snaptb-1699592108764 to region=428db5f998f281799ebc91a16e0c51d8 table=testCompactionWithHFileLink 2023-11-10 04:55:23,599 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a63356bdf54c4d7b9c03c16621133a28 of snapshot=snaptb-1699592108764 to region=ee1662c8ccd6e1d2468c8603ba66a3a6 table=testCompactionWithHFileLink 2023-11-10 04:55:23,602 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c07b5707fdaa439788a90b8039e6f558 of snapshot=snaptb-1699592108764 to region=247092fa363b36c6fef420107a683723 table=testCompactionWithHFileLink 2023-11-10 04:55:23,602 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3dfe247256a54b92a5549e8aa06c7a79 of snapshot=snaptb-1699592108764 to region=ee1662c8ccd6e1d2468c8603ba66a3a6 table=testCompactionWithHFileLink 2023-11-10 04:55:23,603 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a20efa896262435186ccb4afda7dcaa0 of snapshot=snaptb-1699592108764 to region=428db5f998f281799ebc91a16e0c51d8 table=testCompactionWithHFileLink 2023-11-10 04:55:23,606 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1699592108764" table: "testCompactionWithHFileLink" creation_time: 1699592108775 type: FLUSH version: 2 ttl: 0 2023-11-10 04:55:23,607 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1699592123607"}]},"ts":"9223372036854775807"} 2023-11-10 04:55:23,608 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1699592123607"}]},"ts":"9223372036854775807"} 2023-11-10 04:55:23,608 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1699592123607"}]},"ts":"9223372036854775807"} 2023-11-10 04:55:23,609 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-10 04:55:23,609 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 247092fa363b36c6fef420107a683723, NAME => 'testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 428db5f998f281799ebc91a16e0c51d8, NAME => 'testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => ee1662c8ccd6e1d2468c8603ba66a3a6, NAME => 'testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.', STARTKEY => '2', ENDKEY => ''}] 2023-11-10 04:55:23,609 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592123608"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592123608"}]},"ts":"1699592123608"} 2023-11-10 04:55:23,610 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592123608"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592123608"}]},"ts":"1699592123608"} 2023-11-10 04:55:23,610 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592123608"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592123608"}]},"ts":"1699592123608"} 2023-11-10 04:55:23,611 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-10 04:55:23,611 INFO [PEWorker-2] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-11-10 04:55:23,611 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 247092fa363b36c6fef420107a683723, NAME => 'testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 428db5f998f281799ebc91a16e0c51d8, NAME => 'testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => ee1662c8ccd6e1d2468c8603ba66a3a6, NAME => 'testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.', STARTKEY => '2', ENDKEY => ''}] 2023-11-10 04:55:23,612 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1699592108764 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-11-10 04:55:23,613 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1699592108764" table: "testCompactionWithHFileLink" creation_time: 1699592108775 type: FLUSH version: 2 ttl: 0 ) in 529 msec 2023-11-10 04:55:23,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-10 04:55:23,699 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-11-10 04:55:23,699 INFO [Listener at localhost/35765] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1699592108764-1699592120917 2023-11-10 04:55:23,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(751): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1699592108764-1699592120917" 2023-11-10 04:55:23,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] snapshot.SnapshotManager(335): Deleting snapshot: hbase-failsafe-snaptb-1699592108764-1699592120917 2023-11-10 04:55:23,708 INFO [Listener at localhost/35765] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-11-10 04:55:23,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-11-10 04:55:23,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-11-10 04:55:23,724 INFO [PEWorker-3] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-11-10 04:55:23,724 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592123724"}]},"ts":"1699592123724"} 2023-11-10 04:55:23,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-10 04:55:23,725 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-11-10 04:55:23,727 DEBUG [PEWorker-3] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-11-10 04:55:23,727 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, ASSIGN}] 2023-11-10 04:55:23,729 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, ASSIGN 2023-11-10 04:55:23,729 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, ASSIGN 2023-11-10 04:55:23,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, ASSIGN 2023-11-10 04:55:23,729 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-10 04:55:23,729 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-10 04:55:23,729 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-10 04:55:23,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-10 04:55:23,881 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=ee1662c8ccd6e1d2468c8603ba66a3a6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:23,881 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=428db5f998f281799ebc91a16e0c51d8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:23,881 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=247092fa363b36c6fef420107a683723, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:23,881 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592123881"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592123881"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592123881"}]},"ts":"1699592123881"} 2023-11-10 04:55:23,881 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592123881"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592123881"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592123881"}]},"ts":"1699592123881"} 2023-11-10 04:55:23,881 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592123881"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592123881"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592123881"}]},"ts":"1699592123881"} 2023-11-10 04:55:23,883 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure ee1662c8ccd6e1d2468c8603ba66a3a6, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:23,883 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=102, ppid=99, state=RUNNABLE; OpenRegionProcedure 428db5f998f281799ebc91a16e0c51d8, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:23,884 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure 247092fa363b36c6fef420107a683723, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:24,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-10 04:55:24,034 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:24,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:24,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 428db5f998f281799ebc91a16e0c51d8, NAME => 'testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.', STARTKEY => '1', ENDKEY => '2'} 2023-11-10 04:55:24,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:24,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:24,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:24,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:24,038 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:24,039 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1 2023-11-10 04:55:24,039 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1 2023-11-10 04:55:24,039 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 428db5f998f281799ebc91a16e0c51d8 columnFamilyName family1 2023-11-10 04:55:24,048 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-0563518c4e2d4880b8cc81f80d202363 2023-11-10 04:55:24,054 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-54d22b6b43374e0ab599ff9a0a26dabc 2023-11-10 04:55:24,059 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-8a93989bdfc14dea990dbac25df2ee02 2023-11-10 04:55:24,063 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-c6696ef25fb345cbbf0b414f8500fedf 2023-11-10 04:55:24,064 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(310): Store=428db5f998f281799ebc91a16e0c51d8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:24,065 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:24,066 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2 2023-11-10 04:55:24,066 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2 2023-11-10 04:55:24,066 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 428db5f998f281799ebc91a16e0c51d8 columnFamilyName family2 2023-11-10 04:55:24,072 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-4ac76495f51f4d65856ae576d60f9971 2023-11-10 04:55:24,077 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-a20efa896262435186ccb4afda7dcaa0 2023-11-10 04:55:24,081 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ad709bef78f04200a7e5a19bfc072ba1 2023-11-10 04:55:24,085 DEBUG [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ae4b2c41494145ffab0e5a5e9c571e89 2023-11-10 04:55:24,085 INFO [StoreOpener-428db5f998f281799ebc91a16e0c51d8-1] regionserver.HStore(310): Store=428db5f998f281799ebc91a16e0c51d8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:24,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:24,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:24,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:24,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:55:24,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 428db5f998f281799ebc91a16e0c51d8; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59774409, jitterRate=-0.10929189622402191}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:24,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:24,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8., pid=102, masterSystemTime=1699592124034 2023-11-10 04:55:24,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:55:24,092 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-10 04:55:24,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:55:24,092 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-10 04:55:24,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:24,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:24,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:24,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 247092fa363b36c6fef420107a683723, NAME => 'testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.', STARTKEY => '', ENDKEY => '1'} 2023-11-10 04:55:24,094 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=428db5f998f281799ebc91a16e0c51d8, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:24,094 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592124094"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592124094"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592124094"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592124094"}]},"ts":"1699592124094"} 2023-11-10 04:55:24,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 247092fa363b36c6fef420107a683723 2023-11-10 04:55:24,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:24,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 247092fa363b36c6fef420107a683723 2023-11-10 04:55:24,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 247092fa363b36c6fef420107a683723 2023-11-10 04:55:24,095 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-10 04:55:24,095 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1912): 428db5f998f281799ebc91a16e0c51d8/family1 is initiating minor compaction (all files) 2023-11-10 04:55:24,095 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-10 04:55:24,095 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 428db5f998f281799ebc91a16e0c51d8/family1 in testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:24,095 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HStore(1912): 428db5f998f281799ebc91a16e0c51d8/family2 is initiating minor compaction (all files) 2023-11-10 04:55:24,095 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-8a93989bdfc14dea990dbac25df2ee02, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-c6696ef25fb345cbbf0b414f8500fedf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-54d22b6b43374e0ab599ff9a0a26dabc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-0563518c4e2d4880b8cc81f80d202363] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp, totalSize=21.6 K 2023-11-10 04:55:24,095 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 428db5f998f281799ebc91a16e0c51d8/family2 in testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:55:24,095 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ae4b2c41494145ffab0e5a5e9c571e89, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ad709bef78f04200a7e5a19bfc072ba1, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-4ac76495f51f4d65856ae576d60f9971, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-a20efa896262435186ccb4afda7dcaa0] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp, totalSize=20.4 K 2023-11-10 04:55:24,096 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 247092fa363b36c6fef420107a683723 2023-11-10 04:55:24,097 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=102, resume processing ppid=99 2023-11-10 04:55:24,097 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=102, ppid=99, state=SUCCESS; OpenRegionProcedure 428db5f998f281799ebc91a16e0c51d8, server=jenkins-hbase4.apache.org,36419,1699592002485 in 213 msec 2023-11-10 04:55:24,097 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-8a93989bdfc14dea990dbac25df2ee02, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102317 2023-11-10 04:55:24,097 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ae4b2c41494145ffab0e5a5e9c571e89, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102317 2023-11-10 04:55:24,098 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1 2023-11-10 04:55:24,098 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1 2023-11-10 04:55:24,098 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-c6696ef25fb345cbbf0b414f8500fedf, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103722 2023-11-10 04:55:24,098 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=428db5f998f281799ebc91a16e0c51d8, ASSIGN in 370 msec 2023-11-10 04:55:24,098 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ad709bef78f04200a7e5a19bfc072ba1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103722 2023-11-10 04:55:24,098 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 247092fa363b36c6fef420107a683723 columnFamilyName family1 2023-11-10 04:55:24,099 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-54d22b6b43374e0ab599ff9a0a26dabc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:24,104 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-4ac76495f51f4d65856ae576d60f9971, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:24,104 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-0563518c4e2d4880b8cc81f80d202363, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:24,104 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-a20efa896262435186ccb4afda7dcaa0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:24,119 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-18fa4d39f76c454b84c3556bc005dec3 2023-11-10 04:55:24,132 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-343bb832e9ac4876816536ca263980ac 2023-11-10 04:55:24,146 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-b2d95126b7b4496c920348e7cc43b2be 2023-11-10 04:55:24,154 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] throttle.PressureAwareThroughputController(145): 428db5f998f281799ebc91a16e0c51d8#family2#compaction#113 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:24,155 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-f76926bedfc44a90aae90e75491d98dd 2023-11-10 04:55:24,158 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(310): Store=247092fa363b36c6fef420107a683723/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:24,158 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): 428db5f998f281799ebc91a16e0c51d8#family1#compaction#114 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:24,158 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 247092fa363b36c6fef420107a683723 2023-11-10 04:55:24,161 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2 2023-11-10 04:55:24,161 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2 2023-11-10 04:55:24,161 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 247092fa363b36c6fef420107a683723 columnFamilyName family2 2023-11-10 04:55:24,170 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-097cfe0f69164ac28a93a645edd40a1e 2023-11-10 04:55:24,178 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-135a9c88def64bac9e94d078b28edbbf 2023-11-10 04:55:24,183 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-230ea277da404cf39dc02f7f034f7fd7 2023-11-10 04:55:24,187 DEBUG [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-c07b5707fdaa439788a90b8039e6f558 2023-11-10 04:55:24,187 INFO [StoreOpener-247092fa363b36c6fef420107a683723-1] regionserver.HStore(310): Store=247092fa363b36c6fef420107a683723/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:24,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723 2023-11-10 04:55:24,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723 2023-11-10 04:55:24,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:24,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 247092fa363b36c6fef420107a683723 2023-11-10 04:55:24,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 247092fa363b36c6fef420107a683723; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68215545, jitterRate=0.016490831971168518}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:24,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:24,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723., pid=103, masterSystemTime=1699592124034 2023-11-10 04:55:24,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-10 04:55:24,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-10 04:55:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:24,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:24,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee1662c8ccd6e1d2468c8603ba66a3a6, NAME => 'testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.', STARTKEY => '2', ENDKEY => ''} 2023-11-10 04:55:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:24,194 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=247092fa363b36c6fef420107a683723, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:24,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:24,194 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592124194"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592124194"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592124194"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592124194"}]},"ts":"1699592124194"} 2023-11-10 04:55:24,195 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:24,196 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1 2023-11-10 04:55:24,196 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1 2023-11-10 04:55:24,197 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee1662c8ccd6e1d2468c8603ba66a3a6 columnFamilyName family1 2023-11-10 04:55:24,197 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=103, resume processing ppid=98 2023-11-10 04:55:24,197 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure 247092fa363b36c6fef420107a683723, server=jenkins-hbase4.apache.org,36419,1699592002485 in 312 msec 2023-11-10 04:55:24,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=247092fa363b36c6fef420107a683723, ASSIGN in 470 msec 2023-11-10 04:55:24,202 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-1efd2e6cde824fe8afa345aaa743ec20 2023-11-10 04:55:24,208 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-335c1b58d71a4b0088492a1457fb1527 2023-11-10 04:55:24,214 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-61ea0ea6261a4724adaccf6dc92aa8fa 2023-11-10 04:55:24,220 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-9a7ec7a970b54c7d84a57d195c12f5d5 2023-11-10 04:55:24,220 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(310): Store=ee1662c8ccd6e1d2468c8603ba66a3a6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:24,221 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:24,221 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2 2023-11-10 04:55:24,221 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2 2023-11-10 04:55:24,222 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee1662c8ccd6e1d2468c8603ba66a3a6 columnFamilyName family2 2023-11-10 04:55:24,230 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-3dfe247256a54b92a5549e8aa06c7a79 2023-11-10 04:55:24,236 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-8dd99ea06da34d1aa6b90f0c0cd05ed8 2023-11-10 04:55:24,242 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-a63356bdf54c4d7b9c03c16621133a28 2023-11-10 04:55:24,248 DEBUG [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(539): loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-fdaea769dff24e2f87ae03469c289c42 2023-11-10 04:55:24,248 INFO [StoreOpener-ee1662c8ccd6e1d2468c8603ba66a3a6-1] regionserver.HStore(310): Store=ee1662c8ccd6e1d2468c8603ba66a3a6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:24,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:24,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:24,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:24,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:55:24,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ee1662c8ccd6e1d2468c8603ba66a3a6; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69752199, jitterRate=0.039388760924339294}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:24,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:24,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6., pid=101, masterSystemTime=1699592124034 2023-11-10 04:55:24,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-10 04:55:24,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-10 04:55:24,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:24,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:24,254 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=ee1662c8ccd6e1d2468c8603ba66a3a6, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:24,254 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1699592124254"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592124254"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592124254"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592124254"}]},"ts":"1699592124254"} 2023-11-10 04:55:24,257 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=101, resume processing ppid=100 2023-11-10 04:55:24,257 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure ee1662c8ccd6e1d2468c8603ba66a3a6, server=jenkins-hbase4.apache.org,36419,1699592002485 in 372 msec 2023-11-10 04:55:24,258 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=100, resume processing ppid=97 2023-11-10 04:55:24,258 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ee1662c8ccd6e1d2468c8603ba66a3a6, ASSIGN in 530 msec 2023-11-10 04:55:24,258 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592124258"}]},"ts":"1699592124258"} 2023-11-10 04:55:24,259 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-11-10 04:55:24,261 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-11-10 04:55:24,262 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 547 msec 2023-11-10 04:55:24,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-10 04:55:24,328 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-11-10 04:55:24,347 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:24,356 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:24,363 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:24,364 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:24,372 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-10 04:55:24,388 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-11-10 04:55:24,388 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:24,389 DEBUG [Listener at localhost/35765] zookeeper.ReadOnlyZKClient(139): Connect 0x5663c09b to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:55:24,395 DEBUG [Listener at localhost/35765] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@38876764, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:55:24,395 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:24,395 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:24,395 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:24,697 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.07692307692307693, tune throughput to 53.85 MB/second 2023-11-10 04:55:24,985 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family2/7009a0575da34856ad8afc05c83aa61f as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/7009a0575da34856ad8afc05c83aa61f 2023-11-10 04:55:24,989 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 428db5f998f281799ebc91a16e0c51d8/family2 of 428db5f998f281799ebc91a16e0c51d8 into 7009a0575da34856ad8afc05c83aa61f(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:55:24,989 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:24,989 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8., storeName=428db5f998f281799ebc91a16e0c51d8/family2, priority=12, startTime=1699592124092; duration=0sec 2023-11-10 04:55:24,989 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-10 04:55:24,990 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-10 04:55:24,992 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-10 04:55:24,992 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HStore(1912): 247092fa363b36c6fef420107a683723/family1 is initiating minor compaction (all files) 2023-11-10 04:55:24,992 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 247092fa363b36c6fef420107a683723/family1 in testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:24,992 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-343bb832e9ac4876816536ca263980ac, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-f76926bedfc44a90aae90e75491d98dd, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-b2d95126b7b4496c920348e7cc43b2be, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-18fa4d39f76c454b84c3556bc005dec3] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp, totalSize=21.6 K 2023-11-10 04:55:24,993 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-343bb832e9ac4876816536ca263980ac, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102316 2023-11-10 04:55:24,993 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-f76926bedfc44a90aae90e75491d98dd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103721 2023-11-10 04:55:24,994 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-b2d95126b7b4496c920348e7cc43b2be, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:24,995 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-18fa4d39f76c454b84c3556bc005dec3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:25,007 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] throttle.PressureAwareThroughputController(145): 247092fa363b36c6fef420107a683723#family1#compaction#115 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-10 04:55:25,083 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ae4b2c41494145ffab0e5a5e9c571e89, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ad709bef78f04200a7e5a19bfc072ba1, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-4ac76495f51f4d65856ae576d60f9971, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-a20efa896262435186ccb4afda7dcaa0] to archive 2023-11-10 04:55:25,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:25,086 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ae4b2c41494145ffab0e5a5e9c571e89 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ae4b2c41494145ffab0e5a5e9c571e89 2023-11-10 04:55:25,088 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ad709bef78f04200a7e5a19bfc072ba1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-ad709bef78f04200a7e5a19bfc072ba1 2023-11-10 04:55:25,089 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-4ac76495f51f4d65856ae576d60f9971 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-4ac76495f51f4d65856ae576d60f9971 2023-11-10 04:55:25,090 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-a20efa896262435186ccb4afda7dcaa0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family2/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-a20efa896262435186ccb4afda7dcaa0 2023-11-10 04:55:25,291 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023111074a2736fb4f9488db379dcccbaa9afe5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023111074a2736fb4f9488db379dcccbaa9afe5 2023-11-10 04:55:25,292 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:25,295 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110a09ce6a5ade145e5943ebbfcea91e4d3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231110a09ce6a5ade145e5943ebbfcea91e4d3 2023-11-10 04:55:25,295 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111085470e87283b48ca825093c4a1f8fccd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023111085470e87283b48ca825093c4a1f8fccd 2023-11-10 04:55:25,295 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:25,296 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:25,307 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:55:25,311 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33982, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:55:25,317 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:55:25,320 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59590, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:55:25,331 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110/family1/5579e9cc862648fa962d251ce74fe604 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-11-10 04:55:25,331 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110/family1/a4ff422f31124d55aabc17a349a59e75 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-11-10 04:55:25,332 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110/family1/5579e9cc862648fa962d251ce74fe604}] 2023-11-10 04:55:25,332 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110/family1/a4ff422f31124d55aabc17a349a59e75}] 2023-11-10 04:55:25,333 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110/family1/88f4beeeedd449e6813d63d6df1a5c83 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-11-10 04:55:25,334 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=32 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110/family1/88f4beeeedd449e6813d63d6df1a5c83}] 2023-11-10 04:55:25,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110/family1/5579e9cc862648fa962d251ce74fe604 for inclusion in ee1662c8ccd6e1d2468c8603ba66a3a6/family1 2023-11-10 04:55:25,339 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110/family1/a4ff422f31124d55aabc17a349a59e75 for inclusion in 428db5f998f281799ebc91a16e0c51d8/family1 2023-11-10 04:55:25,339 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110/family1/88f4beeeedd449e6813d63d6df1a5c83 for inclusion in 247092fa363b36c6fef420107a683723/family1 2023-11-10 04:55:25,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-11-10 04:55:25,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-10 04:55:25,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-11-10 04:55:25,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-10 04:55:25,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:25,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:25,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-11-10 04:55:25,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last=1 2023-11-10 04:55:25,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:25,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110/family1/a4ff422f31124d55aabc17a349a59e75 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2tg0ojjeslq35tub09sfidlht1aqas4qv4ksb2bdhtdpqjkq3sps7gf1sme5jq7e/family1/a4ff422f31124d55aabc17a349a59e75 2023-11-10 04:55:25,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110/family1/88f4beeeedd449e6813d63d6df1a5c83 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__empncfibb65keuhrjmt977jaip95ioejk9b5gf02rnmvd45m2p5m0uovtb7ddis7/family1/88f4beeeedd449e6813d63d6df1a5c83 2023-11-10 04:55:25,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110/family1/5579e9cc862648fa962d251ce74fe604 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__vs0e8pnu58d0qaotcc0vthehu0i7neqk3r9c987tn2dom7hjp694etu9j3qc9l8h/family1/5579e9cc862648fa962d251ce74fe604 2023-11-10 04:55:25,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2tg0ojjeslq35tub09sfidlht1aqas4qv4ksb2bdhtdpqjkq3sps7gf1sme5jq7e/family1/a4ff422f31124d55aabc17a349a59e75 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0451d186762545679846f2cbff70906b_SeqId_35_ 2023-11-10 04:55:25,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__vs0e8pnu58d0qaotcc0vthehu0i7neqk3r9c987tn2dom7hjp694etu9j3qc9l8h/family1/5579e9cc862648fa962d251ce74fe604 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/cd92bbc9b5b64f14857e8cc1a13156ba_SeqId_34_ 2023-11-10 04:55:25,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__empncfibb65keuhrjmt977jaip95ioejk9b5gf02rnmvd45m2p5m0uovtb7ddis7/family1/88f4beeeedd449e6813d63d6df1a5c83 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/79c995653dca4462b478e6d827141b89_SeqId_34_ 2023-11-10 04:55:25,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__vs0e8pnu58d0qaotcc0vthehu0i7neqk3r9c987tn2dom7hjp694etu9j3qc9l8h/family1/5579e9cc862648fa962d251ce74fe604 into ee1662c8ccd6e1d2468c8603ba66a3a6/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/cd92bbc9b5b64f14857e8cc1a13156ba_SeqId_34_ - updating store file list. 2023-11-10 04:55:25,348 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2tg0ojjeslq35tub09sfidlht1aqas4qv4ksb2bdhtdpqjkq3sps7gf1sme5jq7e/family1/a4ff422f31124d55aabc17a349a59e75 into 428db5f998f281799ebc91a16e0c51d8/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0451d186762545679846f2cbff70906b_SeqId_35_ - updating store file list. 2023-11-10 04:55:25,348 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__empncfibb65keuhrjmt977jaip95ioejk9b5gf02rnmvd45m2p5m0uovtb7ddis7/family1/88f4beeeedd449e6813d63d6df1a5c83 into 247092fa363b36c6fef420107a683723/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/79c995653dca4462b478e6d827141b89_SeqId_34_ - updating store file list. 2023-11-10 04:55:25,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/cd92bbc9b5b64f14857e8cc1a13156ba_SeqId_34_ into ee1662c8ccd6e1d2468c8603ba66a3a6/family1 2023-11-10 04:55:25,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__vs0e8pnu58d0qaotcc0vthehu0i7neqk3r9c987tn2dom7hjp694etu9j3qc9l8h/family1/5579e9cc862648fa962d251ce74fe604 into ee1662c8ccd6e1d2468c8603ba66a3a6/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/cd92bbc9b5b64f14857e8cc1a13156ba_SeqId_34_) 2023-11-10 04:55:25,355 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/79c995653dca4462b478e6d827141b89_SeqId_34_ into 247092fa363b36c6fef420107a683723/family1 2023-11-10 04:55:25,356 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0451d186762545679846f2cbff70906b_SeqId_35_ into 428db5f998f281799ebc91a16e0c51d8/family1 2023-11-10 04:55:25,356 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__empncfibb65keuhrjmt977jaip95ioejk9b5gf02rnmvd45m2p5m0uovtb7ddis7/family1/88f4beeeedd449e6813d63d6df1a5c83 into 247092fa363b36c6fef420107a683723/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/79c995653dca4462b478e6d827141b89_SeqId_34_) 2023-11-10 04:55:25,356 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2tg0ojjeslq35tub09sfidlht1aqas4qv4ksb2bdhtdpqjkq3sps7gf1sme5jq7e/family1/a4ff422f31124d55aabc17a349a59e75 into 428db5f998f281799ebc91a16e0c51d8/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/0451d186762545679846f2cbff70906b_SeqId_35_) 2023-11-10 04:55:25,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__vs0e8pnu58d0qaotcc0vthehu0i7neqk3r9c987tn2dom7hjp694etu9j3qc9l8h/family1/5579e9cc862648fa962d251ce74fe604 2023-11-10 04:55:25,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__empncfibb65keuhrjmt977jaip95ioejk9b5gf02rnmvd45m2p5m0uovtb7ddis7/family1/88f4beeeedd449e6813d63d6df1a5c83 2023-11-10 04:55:25,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testCompactionWithHFileLink__2tg0ojjeslq35tub09sfidlht1aqas4qv4ksb2bdhtdpqjkq3sps7gf1sme5jq7e/family1/a4ff422f31124d55aabc17a349a59e75 2023-11-10 04:55:25,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:25,361 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:25,361 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:25,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-11-10 04:55:25,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(1912): ee1662c8ccd6e1d2468c8603ba66a3a6/family1 is initiating minor compaction (all files) 2023-11-10 04:55:25,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@68bb5486; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-10 04:55:25,363 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231110aa20f878baf844b0bdeca7e8694461f4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231110aa20f878baf844b0bdeca7e8694461f4 2023-11-10 04:55:25,364 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:25,365 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231110b0d110932bae45359e8a228021cd70f1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231110b0d110932bae45359e8a228021cd70f1 2023-11-10 04:55:25,365 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:25,365 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:25,366 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:25,366 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231110d2c8fc62e00a4510a1d823cffe6d60a4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231110d2c8fc62e00a4510a1d823cffe6d60a4 2023-11-10 04:55:25,367 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023111000281f5ebd834e8f9eb29aa07846bab0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023111000281f5ebd834e8f9eb29aa07846bab0 2023-11-10 04:55:25,367 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311106b48e94e65554c4893ee5e1202c8783c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311106b48e94e65554c4893ee5e1202c8783c 2023-11-10 04:55:25,368 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231110ed18e5da3f2348169787daf780e530cd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231110ed18e5da3f2348169787daf780e530cd 2023-11-10 04:55:25,368 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-10 04:55:25,368 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231110' 2023-11-10 04:55:25,368 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:25,368 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231110a49554fc563041f8a8a1665675a6885b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231110a49554fc563041f8a8a1665675a6885b 2023-11-10 04:55:25,369 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231110778980a6a03548eaa0eb0261594b0c47 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231110778980a6a03548eaa0eb0261594b0c47 2023-11-10 04:55:25,370 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231110b0ceb0985d4447449efb0ad15104d7fd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231110b0ceb0985d4447449efb0ad15104d7fd 2023-11-10 04:55:25,370 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311108a9f8f2e160e4ae297ff5d6758f35d78 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311108a9f8f2e160e4ae297ff5d6758f35d78 2023-11-10 04:55:25,371 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231110cefd0f548db540a78c386a0d2a56fde4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20231110cefd0f548db540a78c386a0d2a56fde4 2023-11-10 04:55:25,371 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-10 04:55:25,371 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231110' 2023-11-10 04:55:25,371 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231110efc6c47081fd44c791c6b6fe97a0558f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231110efc6c47081fd44c791c6b6fe97a0558f 2023-11-10 04:55:25,371 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-10 04:55:25,371 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231110' 2023-11-10 04:55:25,371 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:25,371 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231110 for table='testCompactionWithHFileLink' and column='family1' 2023-11-10 04:55:25,371 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-11-10 04:55:25,375 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:25,379 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:25,382 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/.tmp/family1/30f99e5f04d64b248fc8448f19ce53bb as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/30f99e5f04d64b248fc8448f19ce53bb 2023-11-10 04:55:25,383 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:25,384 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:25,387 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 428db5f998f281799ebc91a16e0c51d8/family1 of 428db5f998f281799ebc91a16e0c51d8 into 30f99e5f04d64b248fc8448f19ce53bb(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:25,387 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:55:25,387 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8., storeName=428db5f998f281799ebc91a16e0c51d8/family1, priority=12, startTime=1699592124092; duration=1sec 2023-11-10 04:55:25,387 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-10 04:55:25,387 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ee1662c8ccd6e1d2468c8603ba66a3a6/family1 in testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:25,387 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-335c1b58d71a4b0088492a1457fb1527, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-1efd2e6cde824fe8afa345aaa743ec20, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-9a7ec7a970b54c7d84a57d195c12f5d5, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-61ea0ea6261a4724adaccf6dc92aa8fa, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/cd92bbc9b5b64f14857e8cc1a13156ba_SeqId_34_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp, totalSize=28.9 K 2023-11-10 04:55:25,390 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-335c1b58d71a4b0088492a1457fb1527, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102318 2023-11-10 04:55:25,390 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-1efd2e6cde824fe8afa345aaa743ec20, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103722 2023-11-10 04:55:25,391 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-9a7ec7a970b54c7d84a57d195c12f5d5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:25,391 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-61ea0ea6261a4724adaccf6dc92aa8fa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:25,392 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting cd92bbc9b5b64f14857e8cc1a13156ba_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1699592102318 2023-11-10 04:55:25,394 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:25,397 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:25,414 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): ee1662c8ccd6e1d2468c8603ba66a3a6#family1#compaction#116 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-10 04:55:25,415 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=290 (was 238) - Thread LEAK? -, OpenFileDescriptor=797 (was 621) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=127 (was 124) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=11014 (was 10739) - AvailableMemoryMB LEAK? - 2023-11-10 04:55:25,428 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=296, OpenFileDescriptor=797, MaxFileDescriptor=60000, SystemLoadAverage=127, ProcessCount=167, AvailableMemoryMB=11014 2023-11-10 04:55:25,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-11-10 04:55:25,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-11-10 04:55:25,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-11-10 04:55:25,439 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-10 04:55:25,442 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 9 msec 2023-11-10 04:55:25,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-8a93989bdfc14dea990dbac25df2ee02, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-c6696ef25fb345cbbf0b414f8500fedf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-54d22b6b43374e0ab599ff9a0a26dabc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-0563518c4e2d4880b8cc81f80d202363] to archive 2023-11-10 04:55:25,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:25,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-8a93989bdfc14dea990dbac25df2ee02 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-8a93989bdfc14dea990dbac25df2ee02 2023-11-10 04:55:25,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-c6696ef25fb345cbbf0b414f8500fedf to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-c6696ef25fb345cbbf0b414f8500fedf 2023-11-10 04:55:25,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-54d22b6b43374e0ab599ff9a0a26dabc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-54d22b6b43374e0ab599ff9a0a26dabc 2023-11-10 04:55:25,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-0563518c4e2d4880b8cc81f80d202363 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/family1/testCompactionWithHFileLink=428db5f998f281799ebc91a16e0c51d8-0563518c4e2d4880b8cc81f80d202363 2023-11-10 04:55:25,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-11-10 04:55:25,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-10 04:55:25,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-11-10 04:55:25,542 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:55:25,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-11-10 04:55:25,543 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:55:25,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-10 04:55:25,545 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:25,545 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:25,545 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:25,545 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1 empty. 2023-11-10 04:55:25,545 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d empty. 2023-11-10 04:55:25,545 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9 empty. 2023-11-10 04:55:25,546 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:25,546 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:25,546 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:25,546 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-11-10 04:55:25,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-10 04:55:25,828 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family1/56fcfac3c14f433492ca935e2bb0ef32 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/56fcfac3c14f433492ca935e2bb0ef32 2023-11-10 04:55:25,832 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 247092fa363b36c6fef420107a683723/family1 of 247092fa363b36c6fef420107a683723 into 56fcfac3c14f433492ca935e2bb0ef32(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:55:25,832 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:25,832 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723., storeName=247092fa363b36c6fef420107a683723/family1, priority=12, startTime=1699592124192; duration=0sec 2023-11-10 04:55:25,832 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-10 04:55:25,832 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-10 04:55:25,834 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-10 04:55:25,834 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HStore(1912): 247092fa363b36c6fef420107a683723/family2 is initiating minor compaction (all files) 2023-11-10 04:55:25,834 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 247092fa363b36c6fef420107a683723/family2 in testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:55:25,834 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-097cfe0f69164ac28a93a645edd40a1e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-135a9c88def64bac9e94d078b28edbbf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-230ea277da404cf39dc02f7f034f7fd7, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-c07b5707fdaa439788a90b8039e6f558] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp, totalSize=20.4 K 2023-11-10 04:55:25,835 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-097cfe0f69164ac28a93a645edd40a1e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102316 2023-11-10 04:55:25,835 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-135a9c88def64bac9e94d078b28edbbf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103721 2023-11-10 04:55:25,836 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-230ea277da404cf39dc02f7f034f7fd7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:25,836 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-c07b5707fdaa439788a90b8039e6f558, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:25,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-10 04:55:25,848 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] throttle.PressureAwareThroughputController(145): 247092fa363b36c6fef420107a683723#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-10 04:55:25,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-343bb832e9ac4876816536ca263980ac, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-f76926bedfc44a90aae90e75491d98dd, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-b2d95126b7b4496c920348e7cc43b2be, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-18fa4d39f76c454b84c3556bc005dec3] to archive 2023-11-10 04:55:25,884 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:25,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-343bb832e9ac4876816536ca263980ac to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-343bb832e9ac4876816536ca263980ac 2023-11-10 04:55:25,886 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-f76926bedfc44a90aae90e75491d98dd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-f76926bedfc44a90aae90e75491d98dd 2023-11-10 04:55:25,887 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-b2d95126b7b4496c920348e7cc43b2be to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-b2d95126b7b4496c920348e7cc43b2be 2023-11-10 04:55:25,888 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-18fa4d39f76c454b84c3556bc005dec3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family1/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-18fa4d39f76c454b84c3556bc005dec3 2023-11-10 04:55:25,953 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-11-10 04:55:25,954 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 8ceeb2db4dd461c2eab386dd8a01d75d, NAME => 'ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:25,954 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => cc06acced8f53930ee1704b8e090b9b9, NAME => 'ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:25,954 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => d16400a47b40f4a34c475d374e4521a1, NAME => 'ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:25,960 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:25,961 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 8ceeb2db4dd461c2eab386dd8a01d75d, disabling compactions & flushes 2023-11-10 04:55:25,961 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:25,961 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:25,961 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. after waiting 0 ms 2023-11-10 04:55:25,961 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:25,961 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:25,961 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:26,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-10 04:55:26,311 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:55:26,360 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:26,360 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing cc06acced8f53930ee1704b8e090b9b9, disabling compactions & flushes 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing d16400a47b40f4a34c475d374e4521a1, disabling compactions & flushes 2023-11-10 04:55:26,361 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:26,361 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. after waiting 0 ms 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. after waiting 0 ms 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:26,361 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:26,361 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:26,361 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:26,363 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:55:26,364 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1699592126364"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592126364"}]},"ts":"1699592126364"} 2023-11-10 04:55:26,364 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1699592126364"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592126364"}]},"ts":"1699592126364"} 2023-11-10 04:55:26,364 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1699592126364"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592126364"}]},"ts":"1699592126364"} 2023-11-10 04:55:26,366 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-10 04:55:26,366 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:55:26,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592126366"}]},"ts":"1699592126366"} 2023-11-10 04:55:26,367 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-11-10 04:55:26,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d16400a47b40f4a34c475d374e4521a1, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=cc06acced8f53930ee1704b8e090b9b9, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8ceeb2db4dd461c2eab386dd8a01d75d, ASSIGN}] 2023-11-10 04:55:26,377 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8ceeb2db4dd461c2eab386dd8a01d75d, ASSIGN 2023-11-10 04:55:26,377 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=cc06acced8f53930ee1704b8e090b9b9, ASSIGN 2023-11-10 04:55:26,377 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d16400a47b40f4a34c475d374e4521a1, ASSIGN 2023-11-10 04:55:26,377 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8ceeb2db4dd461c2eab386dd8a01d75d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:26,377 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=cc06acced8f53930ee1704b8e090b9b9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:26,377 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d16400a47b40f4a34c475d374e4521a1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:26,530 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=cc06acced8f53930ee1704b8e090b9b9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:26,530 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d16400a47b40f4a34c475d374e4521a1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:26,530 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=8ceeb2db4dd461c2eab386dd8a01d75d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:26,530 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1699592126530"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592126530"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592126530"}]},"ts":"1699592126530"} 2023-11-10 04:55:26,530 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1699592126530"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592126530"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592126530"}]},"ts":"1699592126530"} 2023-11-10 04:55:26,530 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1699592126530"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592126530"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592126530"}]},"ts":"1699592126530"} 2023-11-10 04:55:26,532 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure cc06acced8f53930ee1704b8e090b9b9, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:26,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure d16400a47b40f4a34c475d374e4521a1, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:26,532 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure 8ceeb2db4dd461c2eab386dd8a01d75d, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:26,633 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family1/b7c68dde3be94c89865371649f021c4b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/b7c68dde3be94c89865371649f021c4b 2023-11-10 04:55:26,637 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in ee1662c8ccd6e1d2468c8603ba66a3a6/family1 of ee1662c8ccd6e1d2468c8603ba66a3a6 into b7c68dde3be94c89865371649f021c4b(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:26,637 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:26,637 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1699592125362; duration=1sec 2023-11-10 04:55:26,638 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-10 04:55:26,638 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-10 04:55:26,639 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-10 04:55:26,639 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1912): ee1662c8ccd6e1d2468c8603ba66a3a6/family2 is initiating minor compaction (all files) 2023-11-10 04:55:26,640 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ee1662c8ccd6e1d2468c8603ba66a3a6/family2 in testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:55:26,640 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-fdaea769dff24e2f87ae03469c289c42, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-8dd99ea06da34d1aa6b90f0c0cd05ed8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-a63356bdf54c4d7b9c03c16621133a28, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-3dfe247256a54b92a5549e8aa06c7a79] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp, totalSize=20.4 K 2023-11-10 04:55:26,640 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-fdaea769dff24e2f87ae03469c289c42, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592102318 2023-11-10 04:55:26,641 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-8dd99ea06da34d1aa6b90f0c0cd05ed8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592103722 2023-11-10 04:55:26,641 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-a63356bdf54c4d7b9c03c16621133a28, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592105532 2023-11-10 04:55:26,642 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-3dfe247256a54b92a5549e8aa06c7a79, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592107350 2023-11-10 04:55:26,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-10 04:55:26,653 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): ee1662c8ccd6e1d2468c8603ba66a3a6#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-10 04:55:26,684 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:26,684 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-335c1b58d71a4b0088492a1457fb1527, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-1efd2e6cde824fe8afa345aaa743ec20, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-9a7ec7a970b54c7d84a57d195c12f5d5, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-61ea0ea6261a4724adaccf6dc92aa8fa, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/cd92bbc9b5b64f14857e8cc1a13156ba_SeqId_34_] to archive 2023-11-10 04:55:26,684 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:26,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:26,686 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-335c1b58d71a4b0088492a1457fb1527 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-335c1b58d71a4b0088492a1457fb1527 2023-11-10 04:55:26,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8ceeb2db4dd461c2eab386dd8a01d75d, NAME => 'ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d.', STARTKEY => '2', ENDKEY => ''} 2023-11-10 04:55:26,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:26,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:26,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:26,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:26,690 INFO [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:26,690 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-1efd2e6cde824fe8afa345aaa743ec20 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-1efd2e6cde824fe8afa345aaa743ec20 2023-11-10 04:55:26,691 DEBUG [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1 2023-11-10 04:55:26,691 DEBUG [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1 2023-11-10 04:55:26,691 INFO [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ceeb2db4dd461c2eab386dd8a01d75d columnFamilyName family1 2023-11-10 04:55:26,691 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-9a7ec7a970b54c7d84a57d195c12f5d5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-9a7ec7a970b54c7d84a57d195c12f5d5 2023-11-10 04:55:26,692 INFO [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] regionserver.HStore(310): Store=8ceeb2db4dd461c2eab386dd8a01d75d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:26,692 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-61ea0ea6261a4724adaccf6dc92aa8fa to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-61ea0ea6261a4724adaccf6dc92aa8fa 2023-11-10 04:55:26,693 INFO [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:26,694 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/cd92bbc9b5b64f14857e8cc1a13156ba_SeqId_34_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family1/cd92bbc9b5b64f14857e8cc1a13156ba_SeqId_34_ 2023-11-10 04:55:26,694 DEBUG [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2 2023-11-10 04:55:26,694 DEBUG [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2 2023-11-10 04:55:26,694 INFO [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ceeb2db4dd461c2eab386dd8a01d75d columnFamilyName family2 2023-11-10 04:55:26,695 INFO [StoreOpener-8ceeb2db4dd461c2eab386dd8a01d75d-1] regionserver.HStore(310): Store=8ceeb2db4dd461c2eab386dd8a01d75d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:26,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:26,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:26,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:26,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:55:26,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:26,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8ceeb2db4dd461c2eab386dd8a01d75d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63595576, jitterRate=-0.052352070808410645}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:26,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:26,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d., pid=111, masterSystemTime=1699592126683 2023-11-10 04:55:26,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:26,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:26,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:26,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cc06acced8f53930ee1704b8e090b9b9, NAME => 'ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9.', STARTKEY => '1', ENDKEY => '2'} 2023-11-10 04:55:26,702 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=8ceeb2db4dd461c2eab386dd8a01d75d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:26,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:26,703 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1699592126702"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592126702"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592126702"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592126702"}]},"ts":"1699592126702"} 2023-11-10 04:55:26,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:26,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:26,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:26,704 INFO [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:26,705 DEBUG [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1 2023-11-10 04:55:26,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=111, resume processing ppid=108 2023-11-10 04:55:26,705 DEBUG [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1 2023-11-10 04:55:26,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure 8ceeb2db4dd461c2eab386dd8a01d75d, server=jenkins-hbase4.apache.org,36419,1699592002485 in 172 msec 2023-11-10 04:55:26,705 INFO [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc06acced8f53930ee1704b8e090b9b9 columnFamilyName family1 2023-11-10 04:55:26,706 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8ceeb2db4dd461c2eab386dd8a01d75d, ASSIGN in 330 msec 2023-11-10 04:55:26,706 INFO [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] regionserver.HStore(310): Store=cc06acced8f53930ee1704b8e090b9b9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:26,707 INFO [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:26,708 DEBUG [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2 2023-11-10 04:55:26,708 DEBUG [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2 2023-11-10 04:55:26,708 INFO [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc06acced8f53930ee1704b8e090b9b9 columnFamilyName family2 2023-11-10 04:55:26,709 INFO [StoreOpener-cc06acced8f53930ee1704b8e090b9b9-1] regionserver.HStore(310): Store=cc06acced8f53930ee1704b8e090b9b9/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:26,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:26,710 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:26,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:26,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:55:26,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:26,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cc06acced8f53930ee1704b8e090b9b9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67351152, jitterRate=0.003610372543334961}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:26,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:26,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9., pid=109, masterSystemTime=1699592126683 2023-11-10 04:55:26,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:26,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:26,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:26,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d16400a47b40f4a34c475d374e4521a1, NAME => 'ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1.', STARTKEY => '', ENDKEY => '1'} 2023-11-10 04:55:26,715 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=cc06acced8f53930ee1704b8e090b9b9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:26,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:26,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:26,715 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1699592126715"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592126715"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592126715"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592126715"}]},"ts":"1699592126715"} 2023-11-10 04:55:26,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:26,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:26,716 INFO [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:26,717 DEBUG [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1 2023-11-10 04:55:26,717 DEBUG [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1 2023-11-10 04:55:26,717 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=109, resume processing ppid=107 2023-11-10 04:55:26,717 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure cc06acced8f53930ee1704b8e090b9b9, server=jenkins-hbase4.apache.org,36419,1699592002485 in 185 msec 2023-11-10 04:55:26,717 INFO [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d16400a47b40f4a34c475d374e4521a1 columnFamilyName family1 2023-11-10 04:55:26,718 INFO [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] regionserver.HStore(310): Store=d16400a47b40f4a34c475d374e4521a1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:26,718 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=cc06acced8f53930ee1704b8e090b9b9, ASSIGN in 342 msec 2023-11-10 04:55:26,718 INFO [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:26,719 DEBUG [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2 2023-11-10 04:55:26,719 DEBUG [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2 2023-11-10 04:55:26,720 INFO [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d16400a47b40f4a34c475d374e4521a1 columnFamilyName family2 2023-11-10 04:55:26,720 INFO [StoreOpener-d16400a47b40f4a34c475d374e4521a1-1] regionserver.HStore(310): Store=d16400a47b40f4a34c475d374e4521a1/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:26,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:26,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:26,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:26,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:55:26,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:26,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d16400a47b40f4a34c475d374e4521a1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74429710, jitterRate=0.10908910632133484}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:26,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:26,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1., pid=110, masterSystemTime=1699592126683 2023-11-10 04:55:26,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:26,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:26,726 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d16400a47b40f4a34c475d374e4521a1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:26,727 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1699592126726"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592126726"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592126726"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592126726"}]},"ts":"1699592126726"} 2023-11-10 04:55:26,728 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=110, resume processing ppid=106 2023-11-10 04:55:26,729 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure d16400a47b40f4a34c475d374e4521a1, server=jenkins-hbase4.apache.org,36419,1699592002485 in 196 msec 2023-11-10 04:55:26,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=106, resume processing ppid=105 2023-11-10 04:55:26,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d16400a47b40f4a34c475d374e4521a1, ASSIGN in 354 msec 2023-11-10 04:55:26,742 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:55:26,742 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592126742"}]},"ts":"1699592126742"} 2023-11-10 04:55:26,742 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:55:26,743 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5663c09b to 127.0.0.1:51169 2023-11-10 04:55:26,743 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:55:26,743 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:55:26,743 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1fad8314 to 127.0.0.1:51169 2023-11-10 04:55:26,743 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:55:26,743 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:55:26,744 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x36829195 to 127.0.0.1:51169 2023-11-10 04:55:26,744 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:55:26,745 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-11-10 04:55:26,748 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:55:26,750 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2080 sec 2023-11-10 04:55:27,063 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/.tmp/family2/9cdbe0d92bd642a4b8fd6d68bad23b94 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/9cdbe0d92bd642a4b8fd6d68bad23b94 2023-11-10 04:55:27,067 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 247092fa363b36c6fef420107a683723/family2 of 247092fa363b36c6fef420107a683723 into 9cdbe0d92bd642a4b8fd6d68bad23b94(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:27,067 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:55:27,067 INFO [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723., storeName=247092fa363b36c6fef420107a683723/family2, priority=12, startTime=1699592124192; duration=1sec 2023-11-10 04:55:27,067 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-10 04:55:27,067 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-11-10 04:55:27,068 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-10 04:55:27,068 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-10 04:55:27,068 DEBUG [RS:0;jenkins-hbase4:36419-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. because compaction request was cancelled 2023-11-10 04:55:27,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-097cfe0f69164ac28a93a645edd40a1e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-135a9c88def64bac9e94d078b28edbbf, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-230ea277da404cf39dc02f7f034f7fd7, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-c07b5707fdaa439788a90b8039e6f558] to archive 2023-11-10 04:55:27,076 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:27,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-097cfe0f69164ac28a93a645edd40a1e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-097cfe0f69164ac28a93a645edd40a1e 2023-11-10 04:55:27,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-135a9c88def64bac9e94d078b28edbbf to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-135a9c88def64bac9e94d078b28edbbf 2023-11-10 04:55:27,080 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-230ea277da404cf39dc02f7f034f7fd7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-230ea277da404cf39dc02f7f034f7fd7 2023-11-10 04:55:27,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-c07b5707fdaa439788a90b8039e6f558 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/family2/testCompactionWithHFileLink=247092fa363b36c6fef420107a683723-c07b5707fdaa439788a90b8039e6f558 2023-11-10 04:55:27,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-10 04:55:27,649 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-11-10 04:55:27,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:27,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:27,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:27,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:27,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-10 04:55:27,659 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-10 04:55:27,659 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:27,659 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-10 04:55:27,660 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-10 04:55:27,660 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:27,660 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:27,662 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:27,662 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:27,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:27,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:27,663 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:27,663 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:27,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:27,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:27,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:27,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:27,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-10 04:55:27,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-10 04:55:27,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-10 04:55:27,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:27,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-10 04:55:27,664 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:27,664 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. started... 2023-11-10 04:55:27,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:27,664 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:27,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:27,666 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d16400a47b40f4a34c475d374e4521a1 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-10 04:55:27,664 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:27,666 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. started... 2023-11-10 04:55:27,666 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. started... 2023-11-10 04:55:27,666 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc06acced8f53930ee1704b8e090b9b9 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-10 04:55:27,666 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8ceeb2db4dd461c2eab386dd8a01d75d 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-10 04:55:27,682 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110d9864680019c444db795d604dda08f6d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231110d9864680019c444db795d604dda08f6d 2023-11-10 04:55:27,683 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111018cc133f4be2406aae502534f8fd4d3c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023111018cc133f4be2406aae502534f8fd4d3c 2023-11-10 04:55:27,871 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/.tmp/family2/6e55bb2aed474099b140b9c482d054cc as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/6e55bb2aed474099b140b9c482d054cc 2023-11-10 04:55:27,875 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in ee1662c8ccd6e1d2468c8603ba66a3a6/family2 of ee1662c8ccd6e1d2468c8603ba66a3a6 into 6e55bb2aed474099b140b9c482d054cc(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:27,875 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:55:27,875 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6., storeName=ee1662c8ccd6e1d2468c8603ba66a3a6/family2, priority=12, startTime=1699592124252; duration=1sec 2023-11-10 04:55:27,875 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:55:27,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-fdaea769dff24e2f87ae03469c289c42, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-8dd99ea06da34d1aa6b90f0c0cd05ed8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-a63356bdf54c4d7b9c03c16621133a28, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-3dfe247256a54b92a5549e8aa06c7a79] to archive 2023-11-10 04:55:27,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:27,881 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-fdaea769dff24e2f87ae03469c289c42 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-fdaea769dff24e2f87ae03469c289c42 2023-11-10 04:55:27,882 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-8dd99ea06da34d1aa6b90f0c0cd05ed8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-8dd99ea06da34d1aa6b90f0c0cd05ed8 2023-11-10 04:55:27,883 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-a63356bdf54c4d7b9c03c16621133a28 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-a63356bdf54c4d7b9c03c16621133a28 2023-11-10 04:55:27,885 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-3dfe247256a54b92a5549e8aa06c7a79 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/family2/testCompactionWithHFileLink=ee1662c8ccd6e1d2468c8603ba66a3a6-3dfe247256a54b92a5549e8aa06c7a79 2023-11-10 04:55:28,083 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023111070f99732e64f4d87aca343ee428ddda8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023111070f99732e64f4d87aca343ee428ddda8 2023-11-10 04:55:28,090 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/7c34e7d1222f4f408e6d6dede510293b 2023-11-10 04:55:28,090 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/7df2ef0a322e42878e3aa4c4c043aff6 2023-11-10 04:55:28,488 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/d2a71429d103464e97043bc1382bfd92 2023-11-10 04:55:28,503 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311108e7b47fd122a4f199d6612c1962c1247 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202311108e7b47fd122a4f199d6612c1962c1247 2023-11-10 04:55:28,504 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023111010bf59988b4b48bd82435699caa7f40e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023111010bf59988b4b48bd82435699caa7f40e 2023-11-10 04:55:28,902 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311104b77e79cc0d64370943a7f10bf4b732a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202311104b77e79cc0d64370943a7f10bf4b732a 2023-11-10 04:55:28,908 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/95d373487bd7496293c117b82346ec84 2023-11-10 04:55:28,911 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/7c34e7d1222f4f408e6d6dede510293b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/7c34e7d1222f4f408e6d6dede510293b 2023-11-10 04:55:28,914 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/7c34e7d1222f4f408e6d6dede510293b, entries=4, sequenceid=5, filesize=5.4 K 2023-11-10 04:55:28,915 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/95d373487bd7496293c117b82346ec84 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/95d373487bd7496293c117b82346ec84 2023-11-10 04:55:28,916 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/3248a80b784542389c859680346b89f2 2023-11-10 04:55:28,918 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/95d373487bd7496293c117b82346ec84, entries=2, sequenceid=5, filesize=5.1 K 2023-11-10 04:55:28,919 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 8ceeb2db4dd461c2eab386dd8a01d75d in 1253ms, sequenceid=5, compaction requested=false 2023-11-10 04:55:28,919 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-11-10 04:55:28,919 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:28,919 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:28,919 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:28,920 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/7df2ef0a322e42878e3aa4c4c043aff6 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7df2ef0a322e42878e3aa4c4c043aff6 2023-11-10 04:55:28,923 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7df2ef0a322e42878e3aa4c4c043aff6, entries=4, sequenceid=5, filesize=5.4 K 2023-11-10 04:55:28,924 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/3248a80b784542389c859680346b89f2 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/3248a80b784542389c859680346b89f2 2023-11-10 04:55:28,927 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/3248a80b784542389c859680346b89f2, entries=2, sequenceid=5, filesize=5.1 K 2023-11-10 04:55:28,928 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d16400a47b40f4a34c475d374e4521a1 in 1262ms, sequenceid=5, compaction requested=false 2023-11-10 04:55:28,928 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:28,928 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:28,929 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:29,307 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/75f68dcc63ea4c35ac4a5f53607eca10 2023-11-10 04:55:29,311 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/d2a71429d103464e97043bc1382bfd92 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/d2a71429d103464e97043bc1382bfd92 2023-11-10 04:55:29,314 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/d2a71429d103464e97043bc1382bfd92, entries=4, sequenceid=5, filesize=5.4 K 2023-11-10 04:55:29,315 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/75f68dcc63ea4c35ac4a5f53607eca10 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/75f68dcc63ea4c35ac4a5f53607eca10 2023-11-10 04:55:29,318 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/75f68dcc63ea4c35ac4a5f53607eca10, entries=2, sequenceid=5, filesize=5.1 K 2023-11-10 04:55:29,319 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for cc06acced8f53930ee1704b8e090b9b9 in 1653ms, sequenceid=5, compaction requested=false 2023-11-10 04:55:29,319 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:29,319 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:29,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:29,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:29,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-10 04:55:29,319 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-10 04:55:29,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:29,321 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:29,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:29,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:29,321 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:29,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:29,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:29,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:29,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:29,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-10 04:55:29,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d45d6b7[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:29,323 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-10 04:55:29,323 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:29,324 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:29,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:29,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:29,324 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-10 04:55:29,324 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-10 04:55:29,324 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:29,324 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,324 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:29,325 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-10 04:55:29,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:29,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:29,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:29,325 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-10 04:55:29,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:29,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:29,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:29,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:29,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:29,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:29,328 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-10 04:55:29,328 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-10 04:55:29,328 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:29,328 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:29,328 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-10 04:55:29,328 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:29,331 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:29,331 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:29,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:29,331 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:29,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:29,331 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:29,333 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:29,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:29,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:29,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:29,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-10 04:55:29,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:29,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:29,335 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:29,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:29,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:29,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:29,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:29,336 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:29,436 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:29,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:29,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:29,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:29,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:29,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:29,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:29,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-10 04:55:29,442 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-10 04:55:29,442 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:29,442 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-10 04:55:29,442 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-10 04:55:29,442 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:29,442 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:29,444 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:29,444 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:29,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:29,444 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:29,444 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:29,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:29,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:29,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:29,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-10 04:55:29,445 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-10 04:55:29,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-10 04:55:29,445 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-10 04:55:29,445 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:29,445 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-10 04:55:29,445 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:29,445 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. started... 2023-11-10 04:55:29,445 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:29,445 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d16400a47b40f4a34c475d374e4521a1 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-10 04:55:29,445 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:29,445 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:29,445 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:29,446 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. started... 2023-11-10 04:55:29,446 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. started... 2023-11-10 04:55:29,446 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc06acced8f53930ee1704b8e090b9b9 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-10 04:55:29,446 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8ceeb2db4dd461c2eab386dd8a01d75d 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-10 04:55:29,462 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110f091987acf0547a5bd094489e6a2ecf5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231110f091987acf0547a5bd094489e6a2ecf5 2023-11-10 04:55:29,462 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110b8a304002fa54231b01fca504f3fe395 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231110b8a304002fa54231b01fca504f3fe395 2023-11-10 04:55:29,468 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/0b3b59919dcd457a8917b73b314e3776 2023-11-10 04:55:29,483 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023111019c0201c5ec54b0fb29e7c43dd4b35bb to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023111019c0201c5ec54b0fb29e7c43dd4b35bb 2023-11-10 04:55:29,863 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023111098e5cd9d37f24b6faa3e17e62102e2ab to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023111098e5cd9d37f24b6faa3e17e62102e2ab 2023-11-10 04:55:29,869 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/113a235c835f41378447f3a36c2fd868 2023-11-10 04:55:29,889 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/98d2fd24315f4cdfb2f63d2caccc1bf0 2023-11-10 04:55:29,893 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/0b3b59919dcd457a8917b73b314e3776 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/0b3b59919dcd457a8917b73b314e3776 2023-11-10 04:55:29,896 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/0b3b59919dcd457a8917b73b314e3776, entries=4, sequenceid=9, filesize=5.4 K 2023-11-10 04:55:29,897 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/98d2fd24315f4cdfb2f63d2caccc1bf0 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/98d2fd24315f4cdfb2f63d2caccc1bf0 2023-11-10 04:55:29,900 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/98d2fd24315f4cdfb2f63d2caccc1bf0, entries=2, sequenceid=9, filesize=5.1 K 2023-11-10 04:55:29,901 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d16400a47b40f4a34c475d374e4521a1 in 456ms, sequenceid=9, compaction requested=false 2023-11-10 04:55:29,901 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:29,901 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:29,901 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:30,268 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/e7e5f43d033c42a1be1a696c7bf647ce 2023-11-10 04:55:30,281 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311105310845e2a6d4782b67f91975f0c397a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202311105310845e2a6d4782b67f91975f0c397a 2023-11-10 04:55:30,682 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311103cfd39a2a1c147b093beb491f42a02e7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202311103cfd39a2a1c147b093beb491f42a02e7 2023-11-10 04:55:30,685 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/25da361394984360be14388365ccd09c 2023-11-10 04:55:30,690 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/8f16a47fa7a045959cbcb12e90d28609 2023-11-10 04:55:30,692 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/113a235c835f41378447f3a36c2fd868 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/113a235c835f41378447f3a36c2fd868 2023-11-10 04:55:30,694 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/e7e5f43d033c42a1be1a696c7bf647ce as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/e7e5f43d033c42a1be1a696c7bf647ce 2023-11-10 04:55:30,696 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/113a235c835f41378447f3a36c2fd868, entries=4, sequenceid=9, filesize=5.4 K 2023-11-10 04:55:30,696 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/25da361394984360be14388365ccd09c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/25da361394984360be14388365ccd09c 2023-11-10 04:55:30,698 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/e7e5f43d033c42a1be1a696c7bf647ce, entries=4, sequenceid=9, filesize=5.4 K 2023-11-10 04:55:30,698 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/8f16a47fa7a045959cbcb12e90d28609 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/8f16a47fa7a045959cbcb12e90d28609 2023-11-10 04:55:30,700 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/25da361394984360be14388365ccd09c, entries=2, sequenceid=9, filesize=5.1 K 2023-11-10 04:55:30,702 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 8ceeb2db4dd461c2eab386dd8a01d75d in 1256ms, sequenceid=9, compaction requested=false 2023-11-10 04:55:30,702 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:30,702 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:30,702 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:30,702 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/8f16a47fa7a045959cbcb12e90d28609, entries=2, sequenceid=9, filesize=5.1 K 2023-11-10 04:55:30,703 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for cc06acced8f53930ee1704b8e090b9b9 in 1257ms, sequenceid=9, compaction requested=false 2023-11-10 04:55:30,703 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:30,703 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:30,703 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:30,703 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:30,703 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,703 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-10 04:55:30,703 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-10 04:55:30,705 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:30,705 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:30,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:30,705 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:30,705 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:30,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:30,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:30,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:30,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:30,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-10 04:55:30,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ce0e2fe[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:30,707 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-10 04:55:30,707 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:30,709 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:30,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:30,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:30,709 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-10 04:55:30,709 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-10 04:55:30,709 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,709 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:30,709 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:30,710 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-10 04:55:30,710 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,710 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:30,710 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-10 04:55:30,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:30,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:30,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:30,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:30,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:30,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:30,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:30,713 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-10 04:55:30,713 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-10 04:55:30,713 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:30,713 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:30,713 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-10 04:55:30,713 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:30,714 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:30,714 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:30,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:30,714 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:30,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:30,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:30,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:30,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:30,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:30,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:30,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:30,715 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:30,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:30,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:30,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:30,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,716 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:30,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:30,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,719 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:30,719 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:30,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:30,719 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-10 04:55:30,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:30,719 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:30,719 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:30,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-10 04:55:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:30,719 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:30,719 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:30,719 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:30,719 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:30,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:30,719 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:30,720 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:30,720 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:30,819 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:30,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:30,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:30,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:30,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:30,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:30,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:30,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-10 04:55:30,826 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-10 04:55:30,826 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:30,826 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-10 04:55:30,826 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-10 04:55:30,827 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:30,827 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:30,829 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:30,829 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:30,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:30,830 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:30,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:30,830 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:30,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:30,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:30,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:30,831 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-10 04:55:30,831 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-10 04:55:30,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-10 04:55:30,831 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-10 04:55:30,831 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:30,831 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-10 04:55:30,831 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:30,831 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. started... 2023-11-10 04:55:30,831 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d16400a47b40f4a34c475d374e4521a1 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-10 04:55:30,834 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:30,834 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:30,834 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:30,834 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. started... 2023-11-10 04:55:30,834 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:30,834 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. started... 2023-11-10 04:55:30,834 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8ceeb2db4dd461c2eab386dd8a01d75d 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-10 04:55:30,834 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc06acced8f53930ee1704b8e090b9b9 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-10 04:55:30,864 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311104d263b9114a64dd88d276df2585e9f61 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311104d263b9114a64dd88d276df2585e9f61 2023-11-10 04:55:31,265 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023111043c50b8d59554b3988f4348785cabadd to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023111043c50b8d59554b3988f4348785cabadd 2023-11-10 04:55:31,265 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023111034c28807b50a492d81bf25b0ef209a81 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023111034c28807b50a492d81bf25b0ef209a81 2023-11-10 04:55:31,277 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/c90b3bfe2fa84aceab89d7654f8ee9a2 2023-11-10 04:55:31,288 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110c01248c293794b2a93b2b7d910944d28 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231110c01248c293794b2a93b2b7d910944d28 2023-11-10 04:55:31,673 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/4757b8e1b63345569f0e43740ddc82af 2023-11-10 04:55:31,676 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/f58d79b8aab7432599e76d2f8242fe4e 2023-11-10 04:55:31,693 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/99a100b0ce384a6b9d0ae3de1649127a 2023-11-10 04:55:31,696 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/c90b3bfe2fa84aceab89d7654f8ee9a2 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/c90b3bfe2fa84aceab89d7654f8ee9a2 2023-11-10 04:55:31,699 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/c90b3bfe2fa84aceab89d7654f8ee9a2, entries=4, sequenceid=13, filesize=5.4 K 2023-11-10 04:55:31,700 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/99a100b0ce384a6b9d0ae3de1649127a as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/99a100b0ce384a6b9d0ae3de1649127a 2023-11-10 04:55:31,702 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/99a100b0ce384a6b9d0ae3de1649127a, entries=2, sequenceid=13, filesize=5.1 K 2023-11-10 04:55:31,703 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 8ceeb2db4dd461c2eab386dd8a01d75d in 869ms, sequenceid=13, compaction requested=true 2023-11-10 04:55:31,703 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:31,703 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:31,703 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:32,087 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110b24dad9efa474241a8ff95f2c9fba9b3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231110b24dad9efa474241a8ff95f2c9fba9b3 2023-11-10 04:55:32,088 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110c394246e62204da5953474ddcb881fe9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231110c394246e62204da5953474ddcb881fe9 2023-11-10 04:55:32,092 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/bb96b55aef6e42a69932ebfd990b8bab 2023-11-10 04:55:32,093 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/2de7a32d319743ca97aae4672e8bd7f8 2023-11-10 04:55:32,096 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/4757b8e1b63345569f0e43740ddc82af as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/4757b8e1b63345569f0e43740ddc82af 2023-11-10 04:55:32,097 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/f58d79b8aab7432599e76d2f8242fe4e as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/f58d79b8aab7432599e76d2f8242fe4e 2023-11-10 04:55:32,099 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/4757b8e1b63345569f0e43740ddc82af, entries=4, sequenceid=13, filesize=5.4 K 2023-11-10 04:55:32,100 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/bb96b55aef6e42a69932ebfd990b8bab as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/bb96b55aef6e42a69932ebfd990b8bab 2023-11-10 04:55:32,100 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/f58d79b8aab7432599e76d2f8242fe4e, entries=4, sequenceid=13, filesize=5.4 K 2023-11-10 04:55:32,101 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/2de7a32d319743ca97aae4672e8bd7f8 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/2de7a32d319743ca97aae4672e8bd7f8 2023-11-10 04:55:32,103 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/bb96b55aef6e42a69932ebfd990b8bab, entries=2, sequenceid=13, filesize=5.1 K 2023-11-10 04:55:32,104 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d16400a47b40f4a34c475d374e4521a1 in 1273ms, sequenceid=13, compaction requested=true 2023-11-10 04:55:32,104 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:32,104 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:32,104 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:32,104 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/2de7a32d319743ca97aae4672e8bd7f8, entries=2, sequenceid=13, filesize=5.1 K 2023-11-10 04:55:32,105 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for cc06acced8f53930ee1704b8e090b9b9 in 1271ms, sequenceid=13, compaction requested=true 2023-11-10 04:55:32,105 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:32,105 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:32,105 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:32,105 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:32,105 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,105 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-10 04:55:32,105 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-10 04:55:32,107 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,107 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:32,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:32,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:32,107 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:32,107 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:32,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:32,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:32,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:32,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:32,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-10 04:55:32,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25967276[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:32,109 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-10 04:55:32,109 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:32,110 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:32,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:32,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:32,110 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-10 04:55:32,110 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-10 04:55:32,110 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:32,110 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,110 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:32,111 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-10 04:55:32,111 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:32,111 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,111 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-10 04:55:32,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:32,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:32,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:32,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:32,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:32,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:32,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:32,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:32,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-10 04:55:32,114 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-10 04:55:32,114 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:32,114 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:32,114 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-10 04:55:32,114 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:32,119 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:32,119 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:32,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:32,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:32,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:32,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:32,119 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:32,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:32,120 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:32,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:32,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:32,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:32,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:32,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:32,121 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:32,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:32,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:32,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:32,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-10 04:55:32,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:32,123 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:32,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:32,123 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:32,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-10 04:55:32,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:32,124 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:32,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:32,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:32,188 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:55:32,224 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:32,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:32,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:32,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:32,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:32,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:32,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:32,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-10 04:55:32,230 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-10 04:55:32,230 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:32,230 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-10 04:55:32,230 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-10 04:55:32,231 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:32,231 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:32,232 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:32,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:32,232 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:32,233 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:32,233 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:32,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:32,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:32,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:32,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:32,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-10 04:55:32,234 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-10 04:55:32,234 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-10 04:55:32,234 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-10 04:55:32,234 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:32,234 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-10 04:55:32,234 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:32,234 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:32,234 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:32,234 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:32,234 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:32,234 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. started... 2023-11-10 04:55:32,234 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. started... 2023-11-10 04:55:32,234 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. started... 2023-11-10 04:55:32,234 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc06acced8f53930ee1704b8e090b9b9 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-10 04:55:32,235 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d16400a47b40f4a34c475d374e4521a1 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-10 04:55:32,234 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8ceeb2db4dd461c2eab386dd8a01d75d 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-10 04:55:32,262 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110dae91b13699842d0acd3cf3c804da681 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231110dae91b13699842d0acd3cf3c804da681 2023-11-10 04:55:32,263 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110d4373becc3bb4bf6bbc27dd040609d60 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231110d4373becc3bb4bf6bbc27dd040609d60 2023-11-10 04:55:32,661 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111042c1a2ef0e534da68927fec19b80cfb6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023111042c1a2ef0e534da68927fec19b80cfb6 2023-11-10 04:55:32,673 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/32e2c7a48f3c469cb02bb1f71ec0714f 2023-11-10 04:55:32,674 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/7d5107a739114d71a1e620a3b72d2c02 2023-11-10 04:55:32,686 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110b4d920324dbe4ee4935c31b60260be16 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231110b4d920324dbe4ee4935c31b60260be16 2023-11-10 04:55:33,067 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/9c530fc6d23144cc95604903a2d1f1af 2023-11-10 04:55:33,086 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110379573568ec94c2f814d762bac8d72d8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231110379573568ec94c2f814d762bac8d72d8 2023-11-10 04:55:33,090 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/aaf13cc2272f415eb6faad537d9947f6 2023-11-10 04:55:33,091 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/6e909d6fa1014f05b3e8a625037fac14 2023-11-10 04:55:33,094 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/32e2c7a48f3c469cb02bb1f71ec0714f as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/32e2c7a48f3c469cb02bb1f71ec0714f 2023-11-10 04:55:33,094 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/7d5107a739114d71a1e620a3b72d2c02 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7d5107a739114d71a1e620a3b72d2c02 2023-11-10 04:55:33,097 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/32e2c7a48f3c469cb02bb1f71ec0714f, entries=4, sequenceid=17, filesize=5.4 K 2023-11-10 04:55:33,097 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7d5107a739114d71a1e620a3b72d2c02, entries=4, sequenceid=17, filesize=5.4 K 2023-11-10 04:55:33,098 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/aaf13cc2272f415eb6faad537d9947f6 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/aaf13cc2272f415eb6faad537d9947f6 2023-11-10 04:55:33,098 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/6e909d6fa1014f05b3e8a625037fac14 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/6e909d6fa1014f05b3e8a625037fac14 2023-11-10 04:55:33,102 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/aaf13cc2272f415eb6faad537d9947f6, entries=2, sequenceid=17, filesize=5.1 K 2023-11-10 04:55:33,102 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/6e909d6fa1014f05b3e8a625037fac14, entries=2, sequenceid=17, filesize=5.1 K 2023-11-10 04:55:33,102 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for cc06acced8f53930ee1704b8e090b9b9 in 868ms, sequenceid=17, compaction requested=true 2023-11-10 04:55:33,103 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d16400a47b40f4a34c475d374e4521a1 in 868ms, sequenceid=17, compaction requested=true 2023-11-10 04:55:33,103 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:33,103 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:33,103 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:33,103 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:33,103 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:33,103 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:33,481 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311102f72d9e6bf5e4ef082baac1a24ac7b7b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202311102f72d9e6bf5e4ef082baac1a24ac7b7b 2023-11-10 04:55:33,886 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/5972cee45e8e49b1a7e703efb8c7b24b 2023-11-10 04:55:33,890 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/9c530fc6d23144cc95604903a2d1f1af as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/9c530fc6d23144cc95604903a2d1f1af 2023-11-10 04:55:33,893 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/9c530fc6d23144cc95604903a2d1f1af, entries=4, sequenceid=17, filesize=5.4 K 2023-11-10 04:55:33,894 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/5972cee45e8e49b1a7e703efb8c7b24b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/5972cee45e8e49b1a7e703efb8c7b24b 2023-11-10 04:55:33,897 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/5972cee45e8e49b1a7e703efb8c7b24b, entries=2, sequenceid=17, filesize=5.1 K 2023-11-10 04:55:33,898 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 8ceeb2db4dd461c2eab386dd8a01d75d in 1664ms, sequenceid=17, compaction requested=true 2023-11-10 04:55:33,898 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:33,898 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:33,898 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:33,899 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:33,899 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,899 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-10 04:55:33,899 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-10 04:55:33,900 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:33,900 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:33,900 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:33,900 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:33,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:33,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:33,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:33,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:33,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-10 04:55:33,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7106aebf[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:33,902 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-10 04:55:33,902 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:33,905 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:33,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:33,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:33,905 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-10 04:55:33,905 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-10 04:55:33,905 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:33,905 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,905 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:33,906 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-10 04:55:33,906 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:33,906 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,906 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-10 04:55:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:33,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:33,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:33,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:33,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-10 04:55:33,908 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-10 04:55:33,908 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:33,908 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:33,908 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-10 04:55:33,908 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:33,910 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:33,910 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:33,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:33,910 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:33,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:33,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:33,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:33,911 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:33,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:33,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:33,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,912 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:33,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:33,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,914 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,914 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:33,914 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:33,914 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:33,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:33,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:33,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:33,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:33,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:33,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:33,915 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:33,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-10 04:55:33,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:33,915 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:33,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:33,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:33,915 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:33,915 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:34,016 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:34,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:34,027 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:34,037 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:34,043 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:34,044 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:34,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:34,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:34,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-10 04:55:34,071 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-10 04:55:34,071 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:34,071 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-10 04:55:34,071 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-10 04:55:34,071 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:34,071 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:34,073 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:34,073 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:34,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:34,074 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:34,074 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:34,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:34,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:34,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:34,075 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-10 04:55:34,075 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-10 04:55:34,075 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-10 04:55:34,075 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-10 04:55:34,075 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:34,075 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-10 04:55:34,075 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:34,075 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:34,075 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:34,075 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:34,075 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. started... 2023-11-10 04:55:34,075 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. started... 2023-11-10 04:55:34,075 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:34,075 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing cc06acced8f53930ee1704b8e090b9b9 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-10 04:55:34,075 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. started... 2023-11-10 04:55:34,075 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d16400a47b40f4a34c475d374e4521a1 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-10 04:55:34,076 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8ceeb2db4dd461c2eab386dd8a01d75d 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-10 04:55:34,493 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/b71f125ef87d48d09578c53f85b7f5fe 2023-11-10 04:55:34,496 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b71f125ef87d48d09578c53f85b7f5fe 2023-11-10 04:55:34,498 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/4b4430cf6fe745418b65faa8c5915fa6 2023-11-10 04:55:34,501 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b4430cf6fe745418b65faa8c5915fa6 2023-11-10 04:55:34,514 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/b6dc05be44494450914dd7188c8b7b35 2023-11-10 04:55:34,517 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b6dc05be44494450914dd7188c8b7b35 2023-11-10 04:55:34,517 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/4b4430cf6fe745418b65faa8c5915fa6 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/4b4430cf6fe745418b65faa8c5915fa6 2023-11-10 04:55:34,521 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4b4430cf6fe745418b65faa8c5915fa6 2023-11-10 04:55:34,521 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/4b4430cf6fe745418b65faa8c5915fa6, entries=3, sequenceid=23, filesize=5.0 K 2023-11-10 04:55:34,521 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/b6dc05be44494450914dd7188c8b7b35 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/b6dc05be44494450914dd7188c8b7b35 2023-11-10 04:55:34,524 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b6dc05be44494450914dd7188c8b7b35 2023-11-10 04:55:34,524 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/b6dc05be44494450914dd7188c8b7b35, entries=1, sequenceid=23, filesize=5.0 K 2023-11-10 04:55:34,525 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 8ceeb2db4dd461c2eab386dd8a01d75d in 449ms, sequenceid=23, compaction requested=true 2023-11-10 04:55:34,525 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:34,525 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:34,525 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:34,895 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/258486d0199b463f9b25582995b86fbc 2023-11-10 04:55:34,899 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 258486d0199b463f9b25582995b86fbc 2023-11-10 04:55:34,907 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/c5b2ae1f87354050b2d95083a3e573d1 2023-11-10 04:55:34,911 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c5b2ae1f87354050b2d95083a3e573d1 2023-11-10 04:55:34,911 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/b71f125ef87d48d09578c53f85b7f5fe as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/b71f125ef87d48d09578c53f85b7f5fe 2023-11-10 04:55:34,914 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b71f125ef87d48d09578c53f85b7f5fe 2023-11-10 04:55:34,915 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/b71f125ef87d48d09578c53f85b7f5fe, entries=3, sequenceid=23, filesize=5.0 K 2023-11-10 04:55:34,915 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/c5b2ae1f87354050b2d95083a3e573d1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/c5b2ae1f87354050b2d95083a3e573d1 2023-11-10 04:55:34,919 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c5b2ae1f87354050b2d95083a3e573d1 2023-11-10 04:55:34,919 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/c5b2ae1f87354050b2d95083a3e573d1, entries=1, sequenceid=23, filesize=5.0 K 2023-11-10 04:55:34,920 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for cc06acced8f53930ee1704b8e090b9b9 in 845ms, sequenceid=23, compaction requested=true 2023-11-10 04:55:34,920 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:34,920 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:34,920 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:55:35,714 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/ee72d85760e94414ade38ffb74c083d0 2023-11-10 04:55:35,718 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee72d85760e94414ade38ffb74c083d0 2023-11-10 04:55:35,719 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/258486d0199b463f9b25582995b86fbc as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/258486d0199b463f9b25582995b86fbc 2023-11-10 04:55:35,723 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 258486d0199b463f9b25582995b86fbc 2023-11-10 04:55:35,723 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/258486d0199b463f9b25582995b86fbc, entries=3, sequenceid=23, filesize=5.0 K 2023-11-10 04:55:35,723 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/ee72d85760e94414ade38ffb74c083d0 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/ee72d85760e94414ade38ffb74c083d0 2023-11-10 04:55:35,727 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee72d85760e94414ade38ffb74c083d0 2023-11-10 04:55:35,727 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/ee72d85760e94414ade38ffb74c083d0, entries=1, sequenceid=23, filesize=5.0 K 2023-11-10 04:55:35,728 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d16400a47b40f4a34c475d374e4521a1 in 1653ms, sequenceid=23, compaction requested=true 2023-11-10 04:55:35,728 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:35,728 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-10 04:55:35,728 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-10 04:55:35,730 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,730 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:35,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:35,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:35,731 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:35,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:35,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:35,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-10 04:55:35,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72753e46[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:35,732 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-10 04:55:35,732 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:35,733 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:35,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:35,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:35,734 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-10 04:55:35,734 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-10 04:55:35,734 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:35,734 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,734 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:35,735 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-10 04:55:35,735 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,735 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:35,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:35,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:35,735 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-10 04:55:35,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:35,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:35,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:35,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:35,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:35,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:35,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-10 04:55:35,738 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-10 04:55:35,738 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:35,738 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:35,738 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-10 04:55:35,738 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:35,741 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:35,741 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:35,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:35,741 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:35,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:35,742 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:35,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:35,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:35,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:35,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,743 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:35,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-10 04:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,746 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,746 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:35,746 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:35,746 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-10 04:55:35,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:35,746 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:35,746 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:35,746 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:35,746 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:35,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-10 04:55:35,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:35,746 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-10 04:55:35,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-10 04:55:35,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:35,747 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-10 04:55:35,747 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:35,747 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:35,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:35,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:35,847 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:35,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:35,848 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:35,848 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): d16400a47b40f4a34c475d374e4521a1/family1 is initiating major compaction (all files) 2023-11-10 04:55:35,848 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:35,849 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:35,849 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of d16400a47b40f4a34c475d374e4521a1/family1 in ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:35,849 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7df2ef0a322e42878e3aa4c4c043aff6, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/0b3b59919dcd457a8917b73b314e3776, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/4757b8e1b63345569f0e43740ddc82af, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7d5107a739114d71a1e620a3b72d2c02, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/258486d0199b463f9b25582995b86fbc] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp, totalSize=26.6 K 2023-11-10 04:55:35,850 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 7df2ef0a322e42878e3aa4c4c043aff6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592127655 2023-11-10 04:55:35,851 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 0b3b59919dcd457a8917b73b314e3776, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592129437 2023-11-10 04:55:35,851 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 4757b8e1b63345569f0e43740ddc82af, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592130822 2023-11-10 04:55:35,851 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 7d5107a739114d71a1e620a3b72d2c02, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592132226 2023-11-10 04:55:35,852 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 258486d0199b463f9b25582995b86fbc, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:55:35,861 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): d16400a47b40f4a34c475d374e4521a1#family1#compaction#149 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:36,269 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311103b807461482347d78b2474c0dd30a426_del 2023-11-10 04:55:36,269 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311103b807461482347d78b2474c0dd30a426_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311103b807461482347d78b2474c0dd30a426_del 2023-11-10 04:55:37,082 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46e6950282444b6ca837785e54cd83a6 2023-11-10 04:55:37,083 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family1/46e6950282444b6ca837785e54cd83a6 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/46e6950282444b6ca837785e54cd83a6 2023-11-10 04:55:37,086 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46e6950282444b6ca837785e54cd83a6 2023-11-10 04:55:37,087 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d16400a47b40f4a34c475d374e4521a1/family1 of d16400a47b40f4a34c475d374e4521a1 into 46e6950282444b6ca837785e54cd83a6(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:37,087 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:37,087 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:37,087 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): d16400a47b40f4a34c475d374e4521a1/family2 is initiating major compaction (all files) 2023-11-10 04:55:37,087 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:37,088 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:37,088 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of d16400a47b40f4a34c475d374e4521a1/family2 in ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:55:37,088 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/3248a80b784542389c859680346b89f2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/98d2fd24315f4cdfb2f63d2caccc1bf0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/bb96b55aef6e42a69932ebfd990b8bab, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/6e909d6fa1014f05b3e8a625037fac14, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/ee72d85760e94414ade38ffb74c083d0] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp, totalSize=25.3 K 2023-11-10 04:55:37,089 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 3248a80b784542389c859680346b89f2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592127655 2023-11-10 04:55:37,089 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 98d2fd24315f4cdfb2f63d2caccc1bf0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592129437 2023-11-10 04:55:37,089 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting bb96b55aef6e42a69932ebfd990b8bab, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592130822 2023-11-10 04:55:37,090 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 6e909d6fa1014f05b3e8a625037fac14, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592132226 2023-11-10 04:55:37,090 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting ee72d85760e94414ade38ffb74c083d0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:55:37,098 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): d16400a47b40f4a34c475d374e4521a1#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:37,172 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7df2ef0a322e42878e3aa4c4c043aff6, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/0b3b59919dcd457a8917b73b314e3776, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/4757b8e1b63345569f0e43740ddc82af, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7d5107a739114d71a1e620a3b72d2c02, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/258486d0199b463f9b25582995b86fbc] to archive 2023-11-10 04:55:37,173 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:37,175 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7df2ef0a322e42878e3aa4c4c043aff6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7df2ef0a322e42878e3aa4c4c043aff6 2023-11-10 04:55:37,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/0b3b59919dcd457a8917b73b314e3776 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/0b3b59919dcd457a8917b73b314e3776 2023-11-10 04:55:37,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/4757b8e1b63345569f0e43740ddc82af to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/4757b8e1b63345569f0e43740ddc82af 2023-11-10 04:55:37,179 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7d5107a739114d71a1e620a3b72d2c02 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/7d5107a739114d71a1e620a3b72d2c02 2023-11-10 04:55:37,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/258486d0199b463f9b25582995b86fbc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/258486d0199b463f9b25582995b86fbc 2023-11-10 04:55:37,506 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311105dfa4410bf8c49b18fc8cc2481ca6e6f_del 2023-11-10 04:55:37,506 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311105dfa4410bf8c49b18fc8cc2481ca6e6f_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202311105dfa4410bf8c49b18fc8cc2481ca6e6f_del 2023-11-10 04:55:38,320 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 201fed8092f640dabc2c6f0e9dad26f9 2023-11-10 04:55:38,321 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/.tmp/family2/201fed8092f640dabc2c6f0e9dad26f9 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/201fed8092f640dabc2c6f0e9dad26f9 2023-11-10 04:55:38,324 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 201fed8092f640dabc2c6f0e9dad26f9 2023-11-10 04:55:38,325 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d16400a47b40f4a34c475d374e4521a1/family2 of d16400a47b40f4a34c475d374e4521a1 into 201fed8092f640dabc2c6f0e9dad26f9(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:38,325 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:38,325 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:38,326 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): 8ceeb2db4dd461c2eab386dd8a01d75d/family1 is initiating major compaction (all files) 2023-11-10 04:55:38,326 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:38,326 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:38,327 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of 8ceeb2db4dd461c2eab386dd8a01d75d/family1 in ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:38,327 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/7c34e7d1222f4f408e6d6dede510293b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/113a235c835f41378447f3a36c2fd868, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/c90b3bfe2fa84aceab89d7654f8ee9a2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/9c530fc6d23144cc95604903a2d1f1af, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/4b4430cf6fe745418b65faa8c5915fa6] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp, totalSize=26.6 K 2023-11-10 04:55:38,327 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 7c34e7d1222f4f408e6d6dede510293b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592127656 2023-11-10 04:55:38,328 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 113a235c835f41378447f3a36c2fd868, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592129438 2023-11-10 04:55:38,328 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting c90b3bfe2fa84aceab89d7654f8ee9a2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592130822 2023-11-10 04:55:38,328 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 9c530fc6d23144cc95604903a2d1f1af, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592132226 2023-11-10 04:55:38,329 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 4b4430cf6fe745418b65faa8c5915fa6, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:55:38,337 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): 8ceeb2db4dd461c2eab386dd8a01d75d#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:38,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/3248a80b784542389c859680346b89f2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/98d2fd24315f4cdfb2f63d2caccc1bf0, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/bb96b55aef6e42a69932ebfd990b8bab, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/6e909d6fa1014f05b3e8a625037fac14, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/ee72d85760e94414ade38ffb74c083d0] to archive 2023-11-10 04:55:38,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:38,374 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/3248a80b784542389c859680346b89f2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/3248a80b784542389c859680346b89f2 2023-11-10 04:55:38,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/98d2fd24315f4cdfb2f63d2caccc1bf0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/98d2fd24315f4cdfb2f63d2caccc1bf0 2023-11-10 04:55:38,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/bb96b55aef6e42a69932ebfd990b8bab to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/bb96b55aef6e42a69932ebfd990b8bab 2023-11-10 04:55:38,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/6e909d6fa1014f05b3e8a625037fac14 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/6e909d6fa1014f05b3e8a625037fac14 2023-11-10 04:55:38,378 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/ee72d85760e94414ade38ffb74c083d0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family2/ee72d85760e94414ade38ffb74c083d0 2023-11-10 04:55:38,745 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231110930ce5a9ccda42cb91eb10a98717915e_del 2023-11-10 04:55:38,745 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110930ce5a9ccda42cb91eb10a98717915e_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231110930ce5a9ccda42cb91eb10a98717915e_del 2023-11-10 04:55:39,558 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ffaeb996511499b97f1cdc332826d31 2023-11-10 04:55:39,558 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family1/2ffaeb996511499b97f1cdc332826d31 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/2ffaeb996511499b97f1cdc332826d31 2023-11-10 04:55:39,562 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ffaeb996511499b97f1cdc332826d31 2023-11-10 04:55:39,562 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8ceeb2db4dd461c2eab386dd8a01d75d/family1 of 8ceeb2db4dd461c2eab386dd8a01d75d into 2ffaeb996511499b97f1cdc332826d31(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:39,562 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:39,563 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:39,563 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): 8ceeb2db4dd461c2eab386dd8a01d75d/family2 is initiating major compaction (all files) 2023-11-10 04:55:39,563 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:39,563 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:39,563 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of 8ceeb2db4dd461c2eab386dd8a01d75d/family2 in ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:55:39,563 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/95d373487bd7496293c117b82346ec84, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/25da361394984360be14388365ccd09c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/99a100b0ce384a6b9d0ae3de1649127a, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/5972cee45e8e49b1a7e703efb8c7b24b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/b6dc05be44494450914dd7188c8b7b35] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp, totalSize=25.3 K 2023-11-10 04:55:39,563 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 95d373487bd7496293c117b82346ec84, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592127656 2023-11-10 04:55:39,564 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 25da361394984360be14388365ccd09c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592129438 2023-11-10 04:55:39,564 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 99a100b0ce384a6b9d0ae3de1649127a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592130822 2023-11-10 04:55:39,564 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 5972cee45e8e49b1a7e703efb8c7b24b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592132226 2023-11-10 04:55:39,565 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting b6dc05be44494450914dd7188c8b7b35, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:55:39,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/7c34e7d1222f4f408e6d6dede510293b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/113a235c835f41378447f3a36c2fd868, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/c90b3bfe2fa84aceab89d7654f8ee9a2, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/9c530fc6d23144cc95604903a2d1f1af, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/4b4430cf6fe745418b65faa8c5915fa6] to archive 2023-11-10 04:55:39,573 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): 8ceeb2db4dd461c2eab386dd8a01d75d#family2#compaction#152 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:39,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:39,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/7c34e7d1222f4f408e6d6dede510293b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/7c34e7d1222f4f408e6d6dede510293b 2023-11-10 04:55:39,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/113a235c835f41378447f3a36c2fd868 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/113a235c835f41378447f3a36c2fd868 2023-11-10 04:55:39,577 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/c90b3bfe2fa84aceab89d7654f8ee9a2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/c90b3bfe2fa84aceab89d7654f8ee9a2 2023-11-10 04:55:39,578 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/9c530fc6d23144cc95604903a2d1f1af to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/9c530fc6d23144cc95604903a2d1f1af 2023-11-10 04:55:39,579 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/4b4430cf6fe745418b65faa8c5915fa6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/4b4430cf6fe745418b65faa8c5915fa6 2023-11-10 04:55:39,580 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023111025c9deb5cb2145969dc26f85759b131f_del 2023-11-10 04:55:39,580 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111025c9deb5cb2145969dc26f85759b131f_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023111025c9deb5cb2145969dc26f85759b131f_del 2023-11-10 04:55:40,394 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9e6ed69ad6ef4e73a9fa9ddb804894bb 2023-11-10 04:55:40,395 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/.tmp/family2/9e6ed69ad6ef4e73a9fa9ddb804894bb as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/9e6ed69ad6ef4e73a9fa9ddb804894bb 2023-11-10 04:55:40,398 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9e6ed69ad6ef4e73a9fa9ddb804894bb 2023-11-10 04:55:40,399 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8ceeb2db4dd461c2eab386dd8a01d75d/family2 of 8ceeb2db4dd461c2eab386dd8a01d75d into 9e6ed69ad6ef4e73a9fa9ddb804894bb(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:55:40,399 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:40,399 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:40,399 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): cc06acced8f53930ee1704b8e090b9b9/family1 is initiating major compaction (all files) 2023-11-10 04:55:40,399 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:40,399 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:40,400 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of cc06acced8f53930ee1704b8e090b9b9/family1 in ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:40,400 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/d2a71429d103464e97043bc1382bfd92, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/e7e5f43d033c42a1be1a696c7bf647ce, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/f58d79b8aab7432599e76d2f8242fe4e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/32e2c7a48f3c469cb02bb1f71ec0714f, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/b71f125ef87d48d09578c53f85b7f5fe] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp, totalSize=26.6 K 2023-11-10 04:55:40,400 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting d2a71429d103464e97043bc1382bfd92, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592127656 2023-11-10 04:55:40,400 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting e7e5f43d033c42a1be1a696c7bf647ce, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592129438 2023-11-10 04:55:40,401 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting f58d79b8aab7432599e76d2f8242fe4e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592130822 2023-11-10 04:55:40,401 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 32e2c7a48f3c469cb02bb1f71ec0714f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592132226 2023-11-10 04:55:40,401 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting b71f125ef87d48d09578c53f85b7f5fe, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:55:40,409 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): cc06acced8f53930ee1704b8e090b9b9#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:40,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/95d373487bd7496293c117b82346ec84, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/25da361394984360be14388365ccd09c, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/99a100b0ce384a6b9d0ae3de1649127a, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/5972cee45e8e49b1a7e703efb8c7b24b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/b6dc05be44494450914dd7188c8b7b35] to archive 2023-11-10 04:55:40,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:40,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/95d373487bd7496293c117b82346ec84 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/95d373487bd7496293c117b82346ec84 2023-11-10 04:55:40,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/25da361394984360be14388365ccd09c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/25da361394984360be14388365ccd09c 2023-11-10 04:55:40,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/99a100b0ce384a6b9d0ae3de1649127a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/99a100b0ce384a6b9d0ae3de1649127a 2023-11-10 04:55:40,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/5972cee45e8e49b1a7e703efb8c7b24b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/5972cee45e8e49b1a7e703efb8c7b24b 2023-11-10 04:55:40,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/b6dc05be44494450914dd7188c8b7b35 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family2/b6dc05be44494450914dd7188c8b7b35 2023-11-10 04:55:40,818 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231110f12c3d994d874ae3bc966e308662129a_del 2023-11-10 04:55:40,818 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110f12c3d994d874ae3bc966e308662129a_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231110f12c3d994d874ae3bc966e308662129a_del 2023-11-10 04:55:41,632 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 018f883281b04aba9cbf3a9fbdef49b8 2023-11-10 04:55:41,633 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family1/018f883281b04aba9cbf3a9fbdef49b8 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/018f883281b04aba9cbf3a9fbdef49b8 2023-11-10 04:55:41,636 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 018f883281b04aba9cbf3a9fbdef49b8 2023-11-10 04:55:41,637 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cc06acced8f53930ee1704b8e090b9b9/family1 of cc06acced8f53930ee1704b8e090b9b9 into 018f883281b04aba9cbf3a9fbdef49b8(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:55:41,637 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:41,637 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:55:41,637 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): cc06acced8f53930ee1704b8e090b9b9/family2 is initiating major compaction (all files) 2023-11-10 04:55:41,637 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:55:41,637 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:55:41,638 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of cc06acced8f53930ee1704b8e090b9b9/family2 in ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:55:41,638 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/75f68dcc63ea4c35ac4a5f53607eca10, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/8f16a47fa7a045959cbcb12e90d28609, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/2de7a32d319743ca97aae4672e8bd7f8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/aaf13cc2272f415eb6faad537d9947f6, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/c5b2ae1f87354050b2d95083a3e573d1] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp, totalSize=25.3 K 2023-11-10 04:55:41,638 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 75f68dcc63ea4c35ac4a5f53607eca10, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592127656 2023-11-10 04:55:41,638 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 8f16a47fa7a045959cbcb12e90d28609, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592129438 2023-11-10 04:55:41,639 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 2de7a32d319743ca97aae4672e8bd7f8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592130822 2023-11-10 04:55:41,639 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting aaf13cc2272f415eb6faad537d9947f6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592132226 2023-11-10 04:55:41,639 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting c5b2ae1f87354050b2d95083a3e573d1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:55:41,647 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): cc06acced8f53930ee1704b8e090b9b9#family2#compaction#154 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:55:41,654 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231110388fa9604f5e45479236fc0bc5928ace_del 2023-11-10 04:55:41,654 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110388fa9604f5e45479236fc0bc5928ace_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231110388fa9604f5e45479236fc0bc5928ace_del 2023-11-10 04:55:41,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/d2a71429d103464e97043bc1382bfd92, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/e7e5f43d033c42a1be1a696c7bf647ce, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/f58d79b8aab7432599e76d2f8242fe4e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/32e2c7a48f3c469cb02bb1f71ec0714f, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/b71f125ef87d48d09578c53f85b7f5fe] to archive 2023-11-10 04:55:41,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:41,675 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/d2a71429d103464e97043bc1382bfd92 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/d2a71429d103464e97043bc1382bfd92 2023-11-10 04:55:41,677 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/e7e5f43d033c42a1be1a696c7bf647ce to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/e7e5f43d033c42a1be1a696c7bf647ce 2023-11-10 04:55:41,678 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/f58d79b8aab7432599e76d2f8242fe4e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/f58d79b8aab7432599e76d2f8242fe4e 2023-11-10 04:55:41,679 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/32e2c7a48f3c469cb02bb1f71ec0714f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/32e2c7a48f3c469cb02bb1f71ec0714f 2023-11-10 04:55:41,679 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/b71f125ef87d48d09578c53f85b7f5fe to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/b71f125ef87d48d09578c53f85b7f5fe 2023-11-10 04:55:42,066 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c32e6828125476088c649c9ecf1746c 2023-11-10 04:55:42,066 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/.tmp/family2/6c32e6828125476088c649c9ecf1746c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/6c32e6828125476088c649c9ecf1746c 2023-11-10 04:55:42,071 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c32e6828125476088c649c9ecf1746c 2023-11-10 04:55:42,072 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cc06acced8f53930ee1704b8e090b9b9/family2 of cc06acced8f53930ee1704b8e090b9b9 into 6c32e6828125476088c649c9ecf1746c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:55:42,072 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:42,075 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:42,078 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:42,082 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:42,082 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:42,088 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-10 04:55:42,101 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-11-10 04:55:42,104 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311103b807461482347d78b2474c0dd30a426_del 2023-11-10 04:55:42,107 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231110f12c3d994d874ae3bc966e308662129a_del 2023-11-10 04:55:42,110 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231110930ce5a9ccda42cb91eb10a98717915e_del 2023-11-10 04:55:42,110 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-11-10 04:55:42,111 DEBUG [Listener at localhost/35765] zookeeper.ReadOnlyZKClient(139): Connect 0x77e9b0c7 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:55:42,117 DEBUG [Listener at localhost/35765] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1603e256, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:55:42,118 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231110 for table='ns:testMinorCompaction' and column='family1' 2023-11-10 04:55:42,118 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231110 for table='ns:testMinorCompaction' and column='family1' 2023-11-10 04:55:42,118 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231110 for table='ns:testMinorCompaction' and column='family1' 2023-11-10 04:55:42,177 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/75f68dcc63ea4c35ac4a5f53607eca10, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/8f16a47fa7a045959cbcb12e90d28609, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/2de7a32d319743ca97aae4672e8bd7f8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/aaf13cc2272f415eb6faad537d9947f6, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/c5b2ae1f87354050b2d95083a3e573d1] to archive 2023-11-10 04:55:42,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:42,179 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/75f68dcc63ea4c35ac4a5f53607eca10 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/75f68dcc63ea4c35ac4a5f53607eca10 2023-11-10 04:55:42,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/8f16a47fa7a045959cbcb12e90d28609 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/8f16a47fa7a045959cbcb12e90d28609 2023-11-10 04:55:42,181 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/2de7a32d319743ca97aae4672e8bd7f8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/2de7a32d319743ca97aae4672e8bd7f8 2023-11-10 04:55:42,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/aaf13cc2272f415eb6faad537d9947f6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/aaf13cc2272f415eb6faad537d9947f6 2023-11-10 04:55:42,183 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/c5b2ae1f87354050b2d95083a3e573d1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family2/c5b2ae1f87354050b2d95083a3e573d1 2023-11-10 04:55:42,958 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311107e68563d741943e4aa94af9687265df2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311107e68563d741943e4aa94af9687265df2 2023-11-10 04:55:42,959 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='ns:testMinorCompaction' column='family1' 2023-11-10 04:55:42,965 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023111070ec9ab8f40243dea3d6626cf2c1f402 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023111070ec9ab8f40243dea3d6626cf2c1f402 2023-11-10 04:55:42,966 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231110 table='ns:testMinorCompaction' column='family1' 2023-11-10 04:55:42,967 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110c47042ba345b45ac8d475da69e732ed6 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231110c47042ba345b45ac8d475da69e732ed6 2023-11-10 04:55:42,968 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='ns:testMinorCompaction' column='family1' 2023-11-10 04:55:42,975 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:55:42,978 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37880, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:55:42,990 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:55:42,994 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40962, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:55:43,013 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231110/family1/938f8fb5ce534e97ac600b60a2ec4a58 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-11-10 04:55:43,014 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231110/family1/f79d994635a94f30811375e43991fd81 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-11-10 04:55:43,015 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231110/family1/938f8fb5ce534e97ac600b60a2ec4a58}] 2023-11-10 04:55:43,015 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231110/family1/f79d994635a94f30811375e43991fd81}] 2023-11-10 04:55:43,015 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231110/family1/46375cbe2f9143d7bc8b1648c651ba6f first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-11-10 04:55:43,016 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231110/family1/46375cbe2f9143d7bc8b1648c651ba6f}] 2023-11-10 04:55:43,025 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231110/family1/46375cbe2f9143d7bc8b1648c651ba6f for inclusion in 8ceeb2db4dd461c2eab386dd8a01d75d/family1 2023-11-10 04:55:43,025 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231110/family1/938f8fb5ce534e97ac600b60a2ec4a58 for inclusion in d16400a47b40f4a34c475d374e4521a1/family1 2023-11-10 04:55:43,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231110/family1/f79d994635a94f30811375e43991fd81 for inclusion in cc06acced8f53930ee1704b8e090b9b9/family1 2023-11-10 04:55:43,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-11-10 04:55:43,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-10 04:55:43,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:55:43,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-11-10 04:55:43,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-11-10 04:55:43,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last=1 2023-11-10 04:55:43,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-10 04:55:43,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:55:43,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:55:43,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231110/family1/46375cbe2f9143d7bc8b1648c651ba6f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__1eak7m81luncm5elq896hr64acvck52fiipveusjf4ojpvc06cl9ik3i40bacgsk/family1/46375cbe2f9143d7bc8b1648c651ba6f 2023-11-10 04:55:43,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231110/family1/938f8fb5ce534e97ac600b60a2ec4a58 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__qnl07i6i0tnm6ejl7d66av0l7idu9agi2b417ksvd199p9447v6cqa13gkc87f6j/family1/938f8fb5ce534e97ac600b60a2ec4a58 2023-11-10 04:55:43,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231110/family1/f79d994635a94f30811375e43991fd81 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__5pf7tk50phupmcqm16acvo622uamfs1bl6087k8grm1j3pu9hdjd0evjor66dq7e/family1/f79d994635a94f30811375e43991fd81 2023-11-10 04:55:43,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__1eak7m81luncm5elq896hr64acvck52fiipveusjf4ojpvc06cl9ik3i40bacgsk/family1/46375cbe2f9143d7bc8b1648c651ba6f as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/d8342fbd4de24c4c89c3d995a1a2f436_SeqId_28_ 2023-11-10 04:55:43,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__qnl07i6i0tnm6ejl7d66av0l7idu9agi2b417ksvd199p9447v6cqa13gkc87f6j/family1/938f8fb5ce534e97ac600b60a2ec4a58 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/f777aa6af3f64d9f9a48ee32f9a8f6dd_SeqId_28_ 2023-11-10 04:55:43,035 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__1eak7m81luncm5elq896hr64acvck52fiipveusjf4ojpvc06cl9ik3i40bacgsk/family1/46375cbe2f9143d7bc8b1648c651ba6f into 8ceeb2db4dd461c2eab386dd8a01d75d/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/d8342fbd4de24c4c89c3d995a1a2f436_SeqId_28_ - updating store file list. 2023-11-10 04:55:43,035 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__qnl07i6i0tnm6ejl7d66av0l7idu9agi2b417ksvd199p9447v6cqa13gkc87f6j/family1/938f8fb5ce534e97ac600b60a2ec4a58 into d16400a47b40f4a34c475d374e4521a1/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/f777aa6af3f64d9f9a48ee32f9a8f6dd_SeqId_28_ - updating store file list. 2023-11-10 04:55:43,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__5pf7tk50phupmcqm16acvo622uamfs1bl6087k8grm1j3pu9hdjd0evjor66dq7e/family1/f79d994635a94f30811375e43991fd81 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/ff2983e6f93e4878b273558258347b12_SeqId_28_ 2023-11-10 04:55:43,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__5pf7tk50phupmcqm16acvo622uamfs1bl6087k8grm1j3pu9hdjd0evjor66dq7e/family1/f79d994635a94f30811375e43991fd81 into cc06acced8f53930ee1704b8e090b9b9/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/ff2983e6f93e4878b273558258347b12_SeqId_28_ - updating store file list. 2023-11-10 04:55:43,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/d8342fbd4de24c4c89c3d995a1a2f436_SeqId_28_ into 8ceeb2db4dd461c2eab386dd8a01d75d/family1 2023-11-10 04:55:43,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__1eak7m81luncm5elq896hr64acvck52fiipveusjf4ojpvc06cl9ik3i40bacgsk/family1/46375cbe2f9143d7bc8b1648c651ba6f into 8ceeb2db4dd461c2eab386dd8a01d75d/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/family1/d8342fbd4de24c4c89c3d995a1a2f436_SeqId_28_) 2023-11-10 04:55:43,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__1eak7m81luncm5elq896hr64acvck52fiipveusjf4ojpvc06cl9ik3i40bacgsk/family1/46375cbe2f9143d7bc8b1648c651ba6f 2023-11-10 04:55:43,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/ff2983e6f93e4878b273558258347b12_SeqId_28_ into cc06acced8f53930ee1704b8e090b9b9/family1 2023-11-10 04:55:43,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__5pf7tk50phupmcqm16acvo622uamfs1bl6087k8grm1j3pu9hdjd0evjor66dq7e/family1/f79d994635a94f30811375e43991fd81 into cc06acced8f53930ee1704b8e090b9b9/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/family1/ff2983e6f93e4878b273558258347b12_SeqId_28_) 2023-11-10 04:55:43,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__5pf7tk50phupmcqm16acvo622uamfs1bl6087k8grm1j3pu9hdjd0evjor66dq7e/family1/f79d994635a94f30811375e43991fd81 2023-11-10 04:55:43,045 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231110 table='ns:testMinorCompaction' column='family1' 2023-11-10 04:55:43,045 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:43,046 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='ns:testMinorCompaction' column='family1' 2023-11-10 04:55:43,046 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:43,047 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023111070f99732e64f4d87aca343ee428ddda8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023111070f99732e64f4d87aca343ee428ddda8 2023-11-10 04:55:43,047 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-10 04:55:43,047 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231110' 2023-11-10 04:55:43,048 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023111018cc133f4be2406aae502534f8fd4d3c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023111018cc133f4be2406aae502534f8fd4d3c 2023-11-10 04:55:43,048 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-10 04:55:43,048 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231110' 2023-11-10 04:55:43,048 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231110 for table='ns:testMinorCompaction' and column='family1' 2023-11-10 04:55:43,048 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231110 for table='ns:testMinorCompaction' and column='family1' 2023-11-10 04:55:43,051 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/f777aa6af3f64d9f9a48ee32f9a8f6dd_SeqId_28_ into d16400a47b40f4a34c475d374e4521a1/family1 2023-11-10 04:55:43,051 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__qnl07i6i0tnm6ejl7d66av0l7idu9agi2b417ksvd199p9447v6cqa13gkc87f6j/family1/938f8fb5ce534e97ac600b60a2ec4a58 into d16400a47b40f4a34c475d374e4521a1/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/family1/f777aa6af3f64d9f9a48ee32f9a8f6dd_SeqId_28_) 2023-11-10 04:55:43,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__ns_testMinorCompaction__qnl07i6i0tnm6ejl7d66av0l7idu9agi2b417ksvd199p9447v6cqa13gkc87f6j/family1/938f8fb5ce534e97ac600b60a2ec4a58 2023-11-10 04:55:43,053 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='ns:testMinorCompaction' column='family1' 2023-11-10 04:55:43,053 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:43,055 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231110d9864680019c444db795d604dda08f6d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20231110d9864680019c444db795d604dda08f6d 2023-11-10 04:55:43,055 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-10 04:55:43,055 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231110' 2023-11-10 04:55:43,056 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231110 for table='ns:testMinorCompaction' and column='family1' 2023-11-10 04:55:43,056 INFO [Listener at localhost/35765] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-11-10 04:55:43,058 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:43,061 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:43,064 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:55:43,065 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:55:43,088 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=253 (was 296), OpenFileDescriptor=656 (was 797), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=134 (was 127) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=11122 (was 11014) - AvailableMemoryMB LEAK? - 2023-11-10 04:55:43,103 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=253, OpenFileDescriptor=656, MaxFileDescriptor=60000, SystemLoadAverage=134, ProcessCount=167, AvailableMemoryMB=11122 2023-11-10 04:55:43,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-10 04:55:43,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:43,108 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:55:43,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-11-10 04:55:43,109 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:55:43,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-10 04:55:43,110 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863 2023-11-10 04:55:43,110 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863 empty. 2023-11-10 04:55:43,111 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863 2023-11-10 04:55:43,111 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-11-10 04:55:43,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-10 04:55:43,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-10 04:55:43,519 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-10 04:55:43,520 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => d7008324b61f95fffabeee9f0b675863, NAME => 'testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:43,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-10 04:55:43,925 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:43,925 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing d7008324b61f95fffabeee9f0b675863, disabling compactions & flushes 2023-11-10 04:55:43,925 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:43,925 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:43,925 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. after waiting 0 ms 2023-11-10 04:55:43,925 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:43,925 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:43,925 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:43,927 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:55:43,928 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1699592143928"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592143928"}]},"ts":"1699592143928"} 2023-11-10 04:55:43,929 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-10 04:55:43,930 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:55:43,930 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592143930"}]},"ts":"1699592143930"} 2023-11-10 04:55:43,931 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-11-10 04:55:43,935 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d7008324b61f95fffabeee9f0b675863, ASSIGN}] 2023-11-10 04:55:43,936 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d7008324b61f95fffabeee9f0b675863, ASSIGN 2023-11-10 04:55:43,937 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d7008324b61f95fffabeee9f0b675863, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:44,088 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d7008324b61f95fffabeee9f0b675863, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:44,088 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1699592144087"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592144087"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592144087"}]},"ts":"1699592144087"} 2023-11-10 04:55:44,089 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure d7008324b61f95fffabeee9f0b675863, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:44,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-10 04:55:44,240 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:44,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:44,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d7008324b61f95fffabeee9f0b675863, NAME => 'testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863.', STARTKEY => '', ENDKEY => ''} 2023-11-10 04:55:44,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy d7008324b61f95fffabeee9f0b675863 2023-11-10 04:55:44,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:44,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d7008324b61f95fffabeee9f0b675863 2023-11-10 04:55:44,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d7008324b61f95fffabeee9f0b675863 2023-11-10 04:55:44,244 INFO [StoreOpener-d7008324b61f95fffabeee9f0b675863-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d7008324b61f95fffabeee9f0b675863 2023-11-10 04:55:44,245 DEBUG [StoreOpener-d7008324b61f95fffabeee9f0b675863-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1 2023-11-10 04:55:44,246 DEBUG [StoreOpener-d7008324b61f95fffabeee9f0b675863-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1 2023-11-10 04:55:44,246 INFO [StoreOpener-d7008324b61f95fffabeee9f0b675863-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d7008324b61f95fffabeee9f0b675863 columnFamilyName family1 2023-11-10 04:55:44,247 INFO [StoreOpener-d7008324b61f95fffabeee9f0b675863-1] regionserver.HStore(310): Store=d7008324b61f95fffabeee9f0b675863/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:44,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863 2023-11-10 04:55:44,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863 2023-11-10 04:55:44,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d7008324b61f95fffabeee9f0b675863 2023-11-10 04:55:44,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:44,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d7008324b61f95fffabeee9f0b675863; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65025622, jitterRate=-0.03104272484779358}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-10 04:55:44,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:44,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863., pid=114, masterSystemTime=1699592144240 2023-11-10 04:55:44,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:44,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:44,253 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d7008324b61f95fffabeee9f0b675863, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:44,254 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1699592144253"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592144253"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592144253"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592144253"}]},"ts":"1699592144253"} 2023-11-10 04:55:44,256 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=114, resume processing ppid=113 2023-11-10 04:55:44,256 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure d7008324b61f95fffabeee9f0b675863, server=jenkins-hbase4.apache.org,36419,1699592002485 in 166 msec 2023-11-10 04:55:44,257 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=113, resume processing ppid=112 2023-11-10 04:55:44,257 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=d7008324b61f95fffabeee9f0b675863, ASSIGN in 321 msec 2023-11-10 04:55:44,257 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:55:44,257 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592144257"}]},"ts":"1699592144257"} 2023-11-10 04:55:44,258 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-11-10 04:55:44,260 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:55:44,261 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1540 sec 2023-11-10 04:55:45,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-10 04:55:45,214 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-11-10 04:55:45,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:45,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:45,242 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:45,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:45,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:45,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:45,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:45,242 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:45,244 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:45,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:45,244 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:45,244 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:45,244 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:45,245 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:45,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:45,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:45,245 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:45,246 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:45,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:45,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:45,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:45,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:45,246 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:45,246 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:45,247 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:45,247 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-10 04:55:45,667 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130af005bfef7fd4e0387898741f9b83b4c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130af005bfef7fd4e0387898741f9b83b4c 2023-11-10 04:55:46,070 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/98553a9fb8864399a653ee862699ce58 2023-11-10 04:55:46,074 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/98553a9fb8864399a653ee862699ce58 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/98553a9fb8864399a653ee862699ce58 2023-11-10 04:55:46,077 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/98553a9fb8864399a653ee862699ce58, entries=1000, sequenceid=5, filesize=166.0 K 2023-11-10 04:55:46,078 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 831ms, sequenceid=5, compaction requested=false 2023-11-10 04:55:46,078 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:46,079 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:46,079 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:46,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:46,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:46,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:46,079 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:46,084 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:46,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:46,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:46,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:46,085 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:46,085 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bed356b[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:46,085 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:46,086 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,087 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,087 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:46,087 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:46,087 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:46,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:46,088 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:46,088 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,088 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:46,089 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:46,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:46,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:46,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:46,091 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:46,091 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:46,091 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:46,091 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,091 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:46,094 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,094 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:46,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:46,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,094 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:46,095 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:46,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:46,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:46,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:46,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,096 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:46,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,098 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:46,099 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:46,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-10 04:55:46,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:46,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:46,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:46,099 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:46,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:46,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:46,200 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:46,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:46,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:46,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:46,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,206 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:46,206 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:46,206 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:46,206 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:46,206 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,206 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,208 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,208 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:46,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:46,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:46,208 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:46,208 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:46,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:46,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,209 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,209 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:55:46,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:46,209 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:46,209 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:46,209 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:46,209 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:46,209 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:46,209 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:46,209 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:46,209 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-10 04:55:46,621 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112883e138fc234148b0bf037e7da4e288f9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112883e138fc234148b0bf037e7da4e288f9 2023-11-10 04:55:47,026 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/34163440979b4545a13a141a16509b91 2023-11-10 04:55:47,030 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/34163440979b4545a13a141a16509b91 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/34163440979b4545a13a141a16509b91 2023-11-10 04:55:47,034 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/34163440979b4545a13a141a16509b91, entries=1, sequenceid=9, filesize=5.0 K 2023-11-10 04:55:47,035 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 826ms, sequenceid=9, compaction requested=false 2023-11-10 04:55:47,036 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:47,036 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:47,038 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,038 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:47,038 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,038 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:47,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:47,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:47,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3031a15e[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:47,040 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:47,040 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,042 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,042 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:47,042 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:47,042 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:47,042 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,042 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:47,043 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,043 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:47,043 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:47,043 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:47,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:47,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:47,046 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:47,046 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:47,046 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:47,046 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,046 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:47,047 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,047 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:47,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:47,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,047 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:47,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:47,048 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,049 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,051 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,051 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,051 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:47,051 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,051 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:47,051 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-10 04:55:47,051 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:47,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:47,051 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:47,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:47,051 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:47,052 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,052 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:47,052 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:47,052 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:47,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:47,152 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:47,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:47,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:47,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:47,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,158 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:47,158 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:47,158 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:47,158 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:47,159 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,159 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,161 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:47,161 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:47,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:47,162 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,162 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:47,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:47,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,163 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:55:47,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,163 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:47,163 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:47,163 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:47,163 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:47,163 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:47,163 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:47,163 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:47,163 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:55:47,574 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201340129980981465f93078a931f053421 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201340129980981465f93078a931f053421 2023-11-10 04:55:47,979 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/040d04082ba7463f98237cc62f6c289d 2023-11-10 04:55:47,983 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/040d04082ba7463f98237cc62f6c289d as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/040d04082ba7463f98237cc62f6c289d 2023-11-10 04:55:47,987 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/040d04082ba7463f98237cc62f6c289d, entries=1, sequenceid=13, filesize=4.9 K 2023-11-10 04:55:47,987 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 824ms, sequenceid=13, compaction requested=true 2023-11-10 04:55:47,988 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:47,988 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:47,988 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:47,988 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:47,988 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,988 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:47,988 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:47,989 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,989 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:47,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:47,990 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,990 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:47,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:47,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:47,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:47,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:47,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47203459[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:47,991 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:47,991 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,993 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:47,993 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,993 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:47,994 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:47,994 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,994 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,994 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:47,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:47,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:47,996 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:47,996 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:47,997 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:47,997 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,997 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:47,999 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,999 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:47,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:47,999 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:48,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:48,000 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,001 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:48,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:48,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:48,006 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,006 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:48,007 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:48,107 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:48,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:48,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:48,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:48,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,113 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:48,113 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:48,113 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:48,113 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:48,113 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,113 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,114 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:48,115 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:48,115 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,115 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:48,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:48,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,116 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,116 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:55:48,116 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:48,116 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:48,116 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:48,116 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:48,116 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:55:48,126 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d684f282b28045d384fe6b5be1349fcb to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205d684f282b28045d384fe6b5be1349fcb 2023-11-10 04:55:48,531 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/911400eda7e74f6b99e83659b0f505bc 2023-11-10 04:55:48,535 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/911400eda7e74f6b99e83659b0f505bc as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/911400eda7e74f6b99e83659b0f505bc 2023-11-10 04:55:48,538 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/911400eda7e74f6b99e83659b0f505bc, entries=1, sequenceid=17, filesize=4.9 K 2023-11-10 04:55:48,539 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 423ms, sequenceid=17, compaction requested=true 2023-11-10 04:55:48,539 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:48,539 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:48,539 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:48,539 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:48,539 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,539 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:48,539 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:48,542 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,542 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:48,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:48,543 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,543 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:48,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:48,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:48,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:48,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:48,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38ae1009[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:48,544 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:48,544 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,545 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,545 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:48,545 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:48,546 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:48,546 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,546 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:48,547 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:48,547 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,547 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:48,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:48,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:48,547 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:48,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:48,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:48,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:48,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:48,549 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:48,549 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:48,549 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:48,549 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:48,549 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,549 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:48,551 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,551 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:48,551 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:48,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:48,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:48,551 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:48,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:48,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,553 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:48,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,555 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,555 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,555 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,555 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:48,555 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,556 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:48,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:48,556 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,556 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:48,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:48,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:48,556 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:48,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-10 04:55:48,556 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:48,556 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:48,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:48,556 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:48,656 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:48,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:48,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:48,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:48,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,662 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:48,662 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:48,662 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:48,662 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:48,662 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,663 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,664 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:48,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:48,664 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:48,664 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:48,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,664 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:48,665 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:48,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,665 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,665 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:55:48,665 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:48,665 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:48,665 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:48,665 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:48,666 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:48,666 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:48,666 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:48,666 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:48,666 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:55:49,080 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228682206625bfa41bb89634527e2dd80b1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228682206625bfa41bb89634527e2dd80b1 2023-11-10 04:55:49,486 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/3b341c965ccf47cd9904147879a495c8 2023-11-10 04:55:49,490 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/3b341c965ccf47cd9904147879a495c8 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/3b341c965ccf47cd9904147879a495c8 2023-11-10 04:55:49,492 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/3b341c965ccf47cd9904147879a495c8, entries=1, sequenceid=21, filesize=4.9 K 2023-11-10 04:55:49,493 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 827ms, sequenceid=21, compaction requested=true 2023-11-10 04:55:49,493 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:49,493 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:49,493 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:49,493 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:49,494 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,494 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:49,494 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:49,495 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,495 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:49,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:49,496 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,496 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:49,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:49,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:49,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:49,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:49,497 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a6a2c1a[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:49,497 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:49,497 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,499 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,499 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:49,499 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:49,499 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:49,499 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,499 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:49,501 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,501 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:49,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:49,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:49,501 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:49,501 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:49,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:49,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:49,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:49,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:49,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:49,503 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:49,504 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:49,504 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:49,504 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,504 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:49,505 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,505 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,505 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:49,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:49,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:49,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:49,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:49,506 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:49,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:49,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,507 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:49,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,509 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,509 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,509 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,509 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,509 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,509 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,509 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,509 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:49,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-10 04:55:49,509 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:49,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:49,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:49,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:49,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:49,509 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:49,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:49,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:49,510 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:49,510 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:49,610 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:49,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:49,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:49,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:49,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,616 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:49,616 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:49,616 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:49,616 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:49,617 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,617 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,618 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:49,618 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:49,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:49,618 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:49,618 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:49,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:49,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,619 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,619 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:55:49,619 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:49,619 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:49,619 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:49,619 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:49,619 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:49,620 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:49,620 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:49,620 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:49,620 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:55:49,780 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:55:50,031 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ee23c83c8ab8452bb6fcd60871fa8ae3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231ee23c83c8ab8452bb6fcd60871fa8ae3 2023-11-10 04:55:50,436 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/2a945a4d024b41b6867e90b951b63b23 2023-11-10 04:55:50,440 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/2a945a4d024b41b6867e90b951b63b23 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2a945a4d024b41b6867e90b951b63b23 2023-11-10 04:55:50,443 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2a945a4d024b41b6867e90b951b63b23, entries=1, sequenceid=25, filesize=4.9 K 2023-11-10 04:55:50,444 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 824ms, sequenceid=25, compaction requested=true 2023-11-10 04:55:50,444 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:50,444 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:50,444 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:50,444 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:50,444 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,444 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:50,444 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:50,446 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,446 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:50,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:50,446 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,446 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:50,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:50,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:50,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:50,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:50,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76c4158[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:50,448 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:50,448 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,449 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,449 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:50,449 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:50,449 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:50,449 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,449 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:50,454 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:50,454 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:50,454 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,454 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:50,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:50,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:50,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:50,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:50,456 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:50,456 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:50,456 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:50,456 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,456 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:50,458 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,458 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,458 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:50,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:50,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:50,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:50,458 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:50,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:50,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:50,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,459 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:50,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:50,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:50,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:50,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:50,462 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:50,462 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:50,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-10 04:55:50,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:50,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:50,562 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:50,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:50,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:50,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:50,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,568 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:50,568 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:50,568 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:50,568 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:50,569 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,569 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,570 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:50,570 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:50,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:50,570 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:50,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,570 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:50,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:55:50,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:50,571 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:50,571 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:50,571 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:50,572 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:50,572 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:50,572 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:50,572 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:50,572 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:55:50,984 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601016b3182e79acc46faab8fedf7013dc311 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601016b3182e79acc46faab8fedf7013dc311 2023-11-10 04:55:51,389 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/c19f21cdae174e84a24100da9acb4e16 2023-11-10 04:55:51,393 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/c19f21cdae174e84a24100da9acb4e16 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/c19f21cdae174e84a24100da9acb4e16 2023-11-10 04:55:51,396 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/c19f21cdae174e84a24100da9acb4e16, entries=1, sequenceid=29, filesize=4.9 K 2023-11-10 04:55:51,396 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 824ms, sequenceid=29, compaction requested=true 2023-11-10 04:55:51,396 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:51,396 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:51,397 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:51,397 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:51,397 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,397 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:51,397 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:51,399 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,399 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:51,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:51,400 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,400 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:51,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:51,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:51,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:51,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:51,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41283f10[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:51,401 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:51,401 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,403 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,403 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:51,403 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:51,403 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:51,403 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,403 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:51,404 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:51,404 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,404 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:51,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:51,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:51,404 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:51,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:51,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:51,407 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:51,407 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:51,407 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:51,407 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,407 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:51,408 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,408 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:51,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:51,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,408 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:51,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:51,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:51,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:51,409 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:51,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,410 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:51,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,413 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,413 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:51,413 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:51,414 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:51,413 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:51,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:51,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:51,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:51,414 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,414 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,414 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,414 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,414 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,414 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:51,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:51,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:51,414 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:51,514 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:51,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:51,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:51,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:51,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,520 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:51,520 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:51,521 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:51,521 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:51,521 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,521 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,522 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,522 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:51,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:51,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:51,523 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:51,523 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:51,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:55:51,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:51,524 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:51,524 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:51,524 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:51,524 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:51,524 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:51,524 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:51,524 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:51,524 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:55:51,935 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038a3a22f3d0ac4265895366d7871610ab to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601038a3a22f3d0ac4265895366d7871610ab 2023-11-10 04:55:52,340 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/2848c09783d94c33a97de09a55f034de 2023-11-10 04:55:52,344 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/2848c09783d94c33a97de09a55f034de as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2848c09783d94c33a97de09a55f034de 2023-11-10 04:55:52,347 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2848c09783d94c33a97de09a55f034de, entries=1, sequenceid=33, filesize=4.9 K 2023-11-10 04:55:52,347 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 823ms, sequenceid=33, compaction requested=true 2023-11-10 04:55:52,347 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:52,347 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:52,348 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:52,348 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:52,348 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,348 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:52,348 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:52,349 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,349 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:52,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:52,350 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,350 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:52,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:52,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:52,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:52,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:52,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9ce5164[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:52,351 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:52,351 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,352 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,352 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:52,352 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:52,352 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:52,353 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,353 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:52,355 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:52,355 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:52,355 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,355 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:52,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:52,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:52,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:52,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:52,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:52,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:52,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:52,357 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:52,357 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:52,357 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:52,357 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,357 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:52,359 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,359 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,359 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:52,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:52,359 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:52,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:52,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:52,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,361 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:52,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,363 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,363 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,363 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:52,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:52,363 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,363 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:52,363 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,363 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,363 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,363 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:52,364 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:52,365 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:52,365 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:52,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:52,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:52,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:52,464 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:52,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:52,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:52,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:52,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,470 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:52,470 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:52,470 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:52,470 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:52,471 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,471 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,472 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:52,472 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:52,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:52,473 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:52,473 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,474 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:55:52,474 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:52,474 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:52,474 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:52,474 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:52,474 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:52,474 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:52,474 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:52,474 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:52,474 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:55:52,885 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090734c0fac2168340a8beed05f16607fc2b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015090734c0fac2168340a8beed05f16607fc2b 2023-11-10 04:55:53,289 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/0f5ae2c2af98475392150aba9b7e0404 2023-11-10 04:55:53,293 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/0f5ae2c2af98475392150aba9b7e0404 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/0f5ae2c2af98475392150aba9b7e0404 2023-11-10 04:55:53,296 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/0f5ae2c2af98475392150aba9b7e0404, entries=1, sequenceid=37, filesize=4.9 K 2023-11-10 04:55:53,296 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 822ms, sequenceid=37, compaction requested=true 2023-11-10 04:55:53,297 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:53,297 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:53,297 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:53,297 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:53,297 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,297 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:53,297 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:53,302 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,302 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:53,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:53,302 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,302 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:53,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:53,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:53,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:53,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:53,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62afa6bb[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:53,304 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:53,304 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,305 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,305 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:53,305 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:53,305 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:53,305 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,305 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:53,307 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:53,307 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,307 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:53,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:53,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:53,307 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:53,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:53,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:53,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:53,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:53,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:53,310 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:53,310 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:53,310 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:53,310 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,310 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:53,311 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,311 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:53,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,311 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:53,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:53,312 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:53,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,313 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:53,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,315 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,315 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,315 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,316 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:53,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:53,316 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-10 04:55:53,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:53,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:53,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:53,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:53,316 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:53,316 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:53,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:53,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:53,316 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:53,416 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:53,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:53,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:53,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-10 04:55:53,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,423 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:53,423 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:53,423 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-10 04:55:53,423 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-10 04:55:53,423 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,424 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,425 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:53,425 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:53,425 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:53,425 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-10 04:55:53,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:53,426 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-10 04:55:53,426 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:53,426 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-10 04:55:53,426 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-10 04:55:53,426 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:53,427 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:53,427 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. started... 2023-11-10 04:55:53,427 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d7008324b61f95fffabeee9f0b675863 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-10 04:55:53,837 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511208be29862a4394020aea4cb5e32ae955f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511208be29862a4394020aea4cb5e32ae955f 2023-11-10 04:55:54,242 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/fbd3a35842154ec09d26535c2b015b6b 2023-11-10 04:55:54,248 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/fbd3a35842154ec09d26535c2b015b6b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/fbd3a35842154ec09d26535c2b015b6b 2023-11-10 04:55:54,251 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/fbd3a35842154ec09d26535c2b015b6b, entries=1, sequenceid=41, filesize=4.9 K 2023-11-10 04:55:54,252 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d7008324b61f95fffabeee9f0b675863 in 825ms, sequenceid=41, compaction requested=true 2023-11-10 04:55:54,252 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:54,252 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:54,252 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-10 04:55:54,252 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-10 04:55:54,252 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,252 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-10 04:55:54,252 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-10 04:55:54,255 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,255 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:54,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:54,255 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,255 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:55:54,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:54,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:54,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:54,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-10 04:55:54,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ac4d37[Count = 0] remaining members to acquire global barrier 2023-11-10 04:55:54,257 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-10 04:55:54,257 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,258 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,258 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-10 04:55:54,258 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-10 04:55:54,258 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:55:54,258 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,258 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:55:54,259 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,259 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-10 04:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:54,259 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:54,259 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-10 04:55:54,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:54,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:54,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:54,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:55:54,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-10 04:55:54,262 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-10 04:55:54,262 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:55:54,262 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:55:54,262 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,262 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:55:54,263 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,263 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,263 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:54,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:55:54,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:55:54,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:55:54,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:54,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:54,264 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:55:54,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,265 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:55:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,268 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,268 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:54,268 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-10 04:55:54,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:55:54,268 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,268 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:55:54,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:54,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:54,268 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:55:54,268 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:54,269 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,269 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-10 04:55:54,269 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:55:54,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:55:54,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:55:54,269 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,269 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,369 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:55:54,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:55:54,371 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-11-10 04:55:54,373 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:55:54,373 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:55:54,374 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:55:54,374 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592754374 2023-11-10 04:55:54,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-10 04:55:54,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-11-10 04:55:54,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:55:54,377 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x239feebf to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:55:54,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@10e5ab68, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:55:54,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:55:55,215 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103fc16cac5a60b44c4a0dcbe7d9a045895 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103fc16cac5a60b44c4a0dcbe7d9a045895 2023-11-10 04:55:55,217 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:55:55,229 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:55:55,231 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33490, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:55:55,236 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:55:55,237 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53306, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:55:55,244 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/80f86ace526f4d15b857ed00f5cc98f2 first=Optional[r5] last=Optional[r6] 2023-11-10 04:55:55,244 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/80f86ace526f4d15b857ed00f5cc98f2}] 2023-11-10 04:55:55,248 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/80f86ace526f4d15b857ed00f5cc98f2 for inclusion in d7008324b61f95fffabeee9f0b675863/family1 2023-11-10 04:55:55,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-11-10 04:55:55,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:55:55,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:55,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/80f86ace526f4d15b857ed00f5cc98f2 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c74m3d4t89l9jlu0k85uhco9rfcf5stop271uq35me6umlgjpq4hspa45f8f6360/family1/80f86ace526f4d15b857ed00f5cc98f2 2023-11-10 04:55:55,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c74m3d4t89l9jlu0k85uhco9rfcf5stop271uq35me6umlgjpq4hspa45f8f6360/family1/80f86ace526f4d15b857ed00f5cc98f2 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/5423590ca7e04ae1a1f00cf34ef27fe7_SeqId_44_ 2023-11-10 04:55:55,254 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c74m3d4t89l9jlu0k85uhco9rfcf5stop271uq35me6umlgjpq4hspa45f8f6360/family1/80f86ace526f4d15b857ed00f5cc98f2 into d7008324b61f95fffabeee9f0b675863/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/5423590ca7e04ae1a1f00cf34ef27fe7_SeqId_44_ - updating store file list. 2023-11-10 04:55:55,257 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/5423590ca7e04ae1a1f00cf34ef27fe7_SeqId_44_ into d7008324b61f95fffabeee9f0b675863/family1 2023-11-10 04:55:55,257 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c74m3d4t89l9jlu0k85uhco9rfcf5stop271uq35me6umlgjpq4hspa45f8f6360/family1/80f86ace526f4d15b857ed00f5cc98f2 into d7008324b61f95fffabeee9f0b675863/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/5423590ca7e04ae1a1f00cf34ef27fe7_SeqId_44_) 2023-11-10 04:55:55,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c74m3d4t89l9jlu0k85uhco9rfcf5stop271uq35me6umlgjpq4hspa45f8f6360/family1/80f86ace526f4d15b857ed00f5cc98f2 2023-11-10 04:55:55,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-10 04:55:55,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-10 04:55:55,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(1912): d7008324b61f95fffabeee9f0b675863/family1 is initiating minor compaction 2023-11-10 04:55:55,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7c3a37b7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:55:55,260 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d7008324b61f95fffabeee9f0b675863/family1 in testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:55,260 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/34163440979b4545a13a141a16509b91, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/040d04082ba7463f98237cc62f6c289d, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/911400eda7e74f6b99e83659b0f505bc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/3b341c965ccf47cd9904147879a495c8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2a945a4d024b41b6867e90b951b63b23, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/c19f21cdae174e84a24100da9acb4e16, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2848c09783d94c33a97de09a55f034de, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/0f5ae2c2af98475392150aba9b7e0404, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/fbd3a35842154ec09d26535c2b015b6b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/5423590ca7e04ae1a1f00cf34ef27fe7_SeqId_44_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp, totalSize=49.7 K 2023-11-10 04:55:55,260 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 34163440979b4545a13a141a16509b91, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-10 04:55:55,261 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 040d04082ba7463f98237cc62f6c289d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-10 04:55:55,261 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 911400eda7e74f6b99e83659b0f505bc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-10 04:55:55,261 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 3b341c965ccf47cd9904147879a495c8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-10 04:55:55,261 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 2a945a4d024b41b6867e90b951b63b23, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-10 04:55:55,261 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting c19f21cdae174e84a24100da9acb4e16, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-10 04:55:55,261 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 2848c09783d94c33a97de09a55f034de, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-10 04:55:55,261 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 0f5ae2c2af98475392150aba9b7e0404, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-10 04:55:55,261 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting fbd3a35842154ec09d26535c2b015b6b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-10 04:55:55,261 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 5423590ca7e04ae1a1f00cf34ef27fe7_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-10 04:55:55,262 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:55:55,262 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:55,264 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601016b3182e79acc46faab8fedf7013dc311 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601016b3182e79acc46faab8fedf7013dc311 2023-11-10 04:55:55,265 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601038a3a22f3d0ac4265895366d7871610ab to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601038a3a22f3d0ac4265895366d7871610ab 2023-11-10 04:55:55,265 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-11-10 04:55:55,265 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-11-10 04:55:55,265 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:55:55,265 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:55:55,293 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): d7008324b61f95fffabeee9f0b675863#family1#compaction#165 average throughput is 0.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-10 04:55:55,713 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311e796b7f266e4b43bcc7b1873902d2ca to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512311e796b7f266e4b43bcc7b1873902d2ca 2023-11-10 04:55:55,714 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:55:55,735 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/302a40b67b904da6ae0c7d5b698bd103 first=Optional[r1] last=Optional[r4] 2023-11-10 04:55:55,736 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/302a40b67b904da6ae0c7d5b698bd103}] 2023-11-10 04:55:55,738 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/302a40b67b904da6ae0c7d5b698bd103 for inclusion in d7008324b61f95fffabeee9f0b675863/family1 2023-11-10 04:55:55,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-11-10 04:55:55,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:55:55,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:55,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/302a40b67b904da6ae0c7d5b698bd103 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__11n9cmkjf0c3b38d38nh2rp09e58v2igffgml5g6jo77bj9q69efi820dscns5ef/family1/302a40b67b904da6ae0c7d5b698bd103 2023-11-10 04:55:55,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__11n9cmkjf0c3b38d38nh2rp09e58v2igffgml5g6jo77bj9q69efi820dscns5ef/family1/302a40b67b904da6ae0c7d5b698bd103 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ce917c69fc9d4ca9a5646ae8059f4768_SeqId_46_ 2023-11-10 04:55:55,742 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__11n9cmkjf0c3b38d38nh2rp09e58v2igffgml5g6jo77bj9q69efi820dscns5ef/family1/302a40b67b904da6ae0c7d5b698bd103 into d7008324b61f95fffabeee9f0b675863/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ce917c69fc9d4ca9a5646ae8059f4768_SeqId_46_ - updating store file list. 2023-11-10 04:55:55,746 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ce917c69fc9d4ca9a5646ae8059f4768_SeqId_46_ into d7008324b61f95fffabeee9f0b675863/family1 2023-11-10 04:55:55,746 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__11n9cmkjf0c3b38d38nh2rp09e58v2igffgml5g6jo77bj9q69efi820dscns5ef/family1/302a40b67b904da6ae0c7d5b698bd103 into d7008324b61f95fffabeee9f0b675863/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ce917c69fc9d4ca9a5646ae8059f4768_SeqId_46_) 2023-11-10 04:55:55,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__11n9cmkjf0c3b38d38nh2rp09e58v2igffgml5g6jo77bj9q69efi820dscns5ef/family1/302a40b67b904da6ae0c7d5b698bd103 2023-11-10 04:55:55,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:55:55,748 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:55,750 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201340129980981465f93078a931f053421 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201340129980981465f93078a931f053421 2023-11-10 04:55:55,751 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205d684f282b28045d384fe6b5be1349fcb to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205d684f282b28045d384fe6b5be1349fcb 2023-11-10 04:55:55,752 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228682206625bfa41bb89634527e2dd80b1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228682206625bfa41bb89634527e2dd80b1 2023-11-10 04:55:55,753 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231ee23c83c8ab8452bb6fcd60871fa8ae3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231ee23c83c8ab8452bb6fcd60871fa8ae3 2023-11-10 04:55:55,753 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-11-10 04:55:55,753 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-11-10 04:55:55,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:55:55,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:55:56,107 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/59b8f650e642410e93116f55a4a044ad as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/59b8f650e642410e93116f55a4a044ad 2023-11-10 04:55:56,112 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d7008324b61f95fffabeee9f0b675863/family1 of d7008324b61f95fffabeee9f0b675863 into 59b8f650e642410e93116f55a4a044ad(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:55:56,112 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:56,112 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1699592155260; duration=0sec 2023-11-10 04:55:56,112 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:55:56,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/34163440979b4545a13a141a16509b91, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/040d04082ba7463f98237cc62f6c289d, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/911400eda7e74f6b99e83659b0f505bc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/3b341c965ccf47cd9904147879a495c8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2a945a4d024b41b6867e90b951b63b23, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/c19f21cdae174e84a24100da9acb4e16, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2848c09783d94c33a97de09a55f034de, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/0f5ae2c2af98475392150aba9b7e0404, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/fbd3a35842154ec09d26535c2b015b6b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/5423590ca7e04ae1a1f00cf34ef27fe7_SeqId_44_] to archive 2023-11-10 04:55:56,183 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:56,184 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a3d96c81480948569d98fc393abd1adf to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130a3d96c81480948569d98fc393abd1adf 2023-11-10 04:55:56,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/34163440979b4545a13a141a16509b91 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/34163440979b4545a13a141a16509b91 2023-11-10 04:55:56,185 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:55:56,186 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/040d04082ba7463f98237cc62f6c289d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/040d04082ba7463f98237cc62f6c289d 2023-11-10 04:55:56,187 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/911400eda7e74f6b99e83659b0f505bc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/911400eda7e74f6b99e83659b0f505bc 2023-11-10 04:55:56,189 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/3b341c965ccf47cd9904147879a495c8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/3b341c965ccf47cd9904147879a495c8 2023-11-10 04:55:56,190 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2a945a4d024b41b6867e90b951b63b23 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2a945a4d024b41b6867e90b951b63b23 2023-11-10 04:55:56,191 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/c19f21cdae174e84a24100da9acb4e16 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/c19f21cdae174e84a24100da9acb4e16 2023-11-10 04:55:56,192 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2848c09783d94c33a97de09a55f034de to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/2848c09783d94c33a97de09a55f034de 2023-11-10 04:55:56,194 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/0f5ae2c2af98475392150aba9b7e0404 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/0f5ae2c2af98475392150aba9b7e0404 2023-11-10 04:55:56,195 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/fbd3a35842154ec09d26535c2b015b6b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/fbd3a35842154ec09d26535c2b015b6b 2023-11-10 04:55:56,196 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/5423590ca7e04ae1a1f00cf34ef27fe7_SeqId_44_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/5423590ca7e04ae1a1f00cf34ef27fe7_SeqId_44_ 2023-11-10 04:55:56,208 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f9766cef975a4d8496e24fefdd2de752 first=Optional[r00] last=Optional[r8] 2023-11-10 04:55:56,208 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f9766cef975a4d8496e24fefdd2de752}] 2023-11-10 04:55:56,210 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f9766cef975a4d8496e24fefdd2de752 for inclusion in d7008324b61f95fffabeee9f0b675863/family1 2023-11-10 04:55:56,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-11-10 04:55:56,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last= 2023-11-10 04:55:56,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:56,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f9766cef975a4d8496e24fefdd2de752 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qk3qv3prda65l6vm7gidek8kbj72lhibvbfrkqpegoqjo7k1a6lhbpllmc5ivbbu/family1/f9766cef975a4d8496e24fefdd2de752 2023-11-10 04:55:56,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qk3qv3prda65l6vm7gidek8kbj72lhibvbfrkqpegoqjo7k1a6lhbpllmc5ivbbu/family1/f9766cef975a4d8496e24fefdd2de752 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ecaa0faafd1d42efb03c910f2ac888e5_SeqId_49_ 2023-11-10 04:55:56,215 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qk3qv3prda65l6vm7gidek8kbj72lhibvbfrkqpegoqjo7k1a6lhbpllmc5ivbbu/family1/f9766cef975a4d8496e24fefdd2de752 into d7008324b61f95fffabeee9f0b675863/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ecaa0faafd1d42efb03c910f2ac888e5_SeqId_49_ - updating store file list. 2023-11-10 04:55:56,218 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ecaa0faafd1d42efb03c910f2ac888e5_SeqId_49_ into d7008324b61f95fffabeee9f0b675863/family1 2023-11-10 04:55:56,218 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qk3qv3prda65l6vm7gidek8kbj72lhibvbfrkqpegoqjo7k1a6lhbpllmc5ivbbu/family1/f9766cef975a4d8496e24fefdd2de752 into d7008324b61f95fffabeee9f0b675863/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ecaa0faafd1d42efb03c910f2ac888e5_SeqId_49_) 2023-11-10 04:55:56,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qk3qv3prda65l6vm7gidek8kbj72lhibvbfrkqpegoqjo7k1a6lhbpllmc5ivbbu/family1/f9766cef975a4d8496e24fefdd2de752 2023-11-10 04:55:56,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-10 04:55:56,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-10 04:55:56,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(1912): d7008324b61f95fffabeee9f0b675863/family1 is initiating minor compaction (all files) 2023-11-10 04:55:56,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@47e0a544; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:55:56,220 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d7008324b61f95fffabeee9f0b675863/family1 in testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:55:56,220 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/98553a9fb8864399a653ee862699ce58, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/59b8f650e642410e93116f55a4a044ad, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ce917c69fc9d4ca9a5646ae8059f4768_SeqId_46_, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ecaa0faafd1d42efb03c910f2ac888e5_SeqId_49_] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp, totalSize=344.3 K 2023-11-10 04:55:56,220 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 98553a9fb8864399a653ee862699ce58, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878827283 2023-11-10 04:55:56,221 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting 59b8f650e642410e93116f55a4a044ad, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621227283 2023-11-10 04:55:56,221 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting ce917c69fc9d4ca9a5646ae8059f4768_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965227283 2023-11-10 04:55:56,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:55:56,221 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] compactions.Compactor(207): Compacting ecaa0faafd1d42efb03c910f2ac888e5_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014827283 2023-11-10 04:55:56,222 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:56,223 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511208be29862a4394020aea4cb5e32ae955f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511208be29862a4394020aea4cb5e32ae955f 2023-11-10 04:55:56,224 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112883e138fc234148b0bf037e7da4e288f9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112883e138fc234148b0bf037e7da4e288f9 2023-11-10 04:55:56,225 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130af005bfef7fd4e0387898741f9b83b4c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130af005bfef7fd4e0387898741f9b83b4c 2023-11-10 04:55:56,225 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-11-10 04:55:56,225 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-10 04:55:56,226 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:55:56,226 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:55:56,226 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-10 04:55:56,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-10 04:55:56,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:55:56,228 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-10 04:55:56,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.8560 sec 2023-11-10 04:55:56,243 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] throttle.PressureAwareThroughputController(145): d7008324b61f95fffabeee9f0b675863#family1#compaction#166 average throughput is 12.97 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-10 04:55:56,440 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=261 (was 253) - Thread LEAK? -, OpenFileDescriptor=680 (was 656) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=127 (was 134), ProcessCount=167 (was 167), AvailableMemoryMB=11116 (was 11122) 2023-11-10 04:55:56,453 INFO [Listener at localhost/35765] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=261, OpenFileDescriptor=680, MaxFileDescriptor=60000, SystemLoadAverage=127, ProcessCount=167, AvailableMemoryMB=11116 2023-11-10 04:55:56,473 DEBUG [Listener at localhost/35765] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-11-10 04:55:56,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10*~.\x01K\xBF@y)\xE7`\xF1!\xAB\xDC\xFA"\x10\xC6V\xB9$\xE13ZJ\xD8C\x15$w\xA04\x17*\x10O\xB1\xF8\xAF\x82\xD7\xA6D\xD7\x93\x07\x85\xB0\x08\x93A2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-10 04:55:56,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure2.ProcedureExecutor(1028): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-11-10 04:55:56,544 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-11-10 04:55:56,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-11-10 04:55:56,545 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-10 04:55:56,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-10 04:55:56,547 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:56,549 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:56,549 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528 empty. 2023-11-10 04:55:56,549 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:56,550 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0 empty. 2023-11-10 04:55:56,550 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:56,550 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901 empty. 2023-11-10 04:55:56,550 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:56,550 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:56,550 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-11-10 04:55:56,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-10 04:55:56,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-10 04:55:56,964 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-11-10 04:55:56,965 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => bc8052c221bbbcbdf08850f6e381c528, NAME => 'testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10*~.\x01K\xBF@y)\xE7`\xF1!\xAB\xDC\xFA"\x10\xC6V\xB9$\xE13ZJ\xD8C\x15$w\xA04\x17*\x10O\xB1\xF8\xAF\x82\xD7\xA6D\xD7\x93\x07\x85\xB0\x08\x93A2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:56,965 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => f3cbc57e6f6fc77d5d5aa5038d824901, NAME => 'testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10*~.\x01K\xBF@y)\xE7`\xF1!\xAB\xDC\xFA"\x10\xC6V\xB9$\xE13ZJ\xD8C\x15$w\xA04\x17*\x10O\xB1\xF8\xAF\x82\xD7\xA6D\xD7\x93\x07\x85\xB0\x08\x93A2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:56,966 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 8b858392c9cfe904206ed137d6282aa0, NAME => 'testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10*~.\x01K\xBF@y)\xE7`\xF1!\xAB\xDC\xFA"\x10\xC6V\xB9$\xE13ZJ\xD8C\x15$w\xA04\x17*\x10O\xB1\xF8\xAF\x82\xD7\xA6D\xD7\x93\x07\x85\xB0\x08\x93A2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/.tmp 2023-11-10 04:55:56,982 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:56,982 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing f3cbc57e6f6fc77d5d5aa5038d824901, disabling compactions & flushes 2023-11-10 04:55:56,982 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:55:56,982 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:55:56,982 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. after waiting 0 ms 2023-11-10 04:55:56,982 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:55:56,982 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:55:56,982 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:55:57,066 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/.tmp/family1/52acc2de88b846cab917dd6131490ef3 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/52acc2de88b846cab917dd6131490ef3 2023-11-10 04:55:57,069 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d7008324b61f95fffabeee9f0b675863/family1 of d7008324b61f95fffabeee9f0b675863 into 52acc2de88b846cab917dd6131490ef3(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:55:57,069 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:55:57,070 INFO [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1699592156220; duration=0sec 2023-11-10 04:55:57,070 DEBUG [RS:0;jenkins-hbase4:36419-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-10 04:55:57,072 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/98553a9fb8864399a653ee862699ce58, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/59b8f650e642410e93116f55a4a044ad, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ce917c69fc9d4ca9a5646ae8059f4768_SeqId_46_, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ecaa0faafd1d42efb03c910f2ac888e5_SeqId_49_] to archive 2023-11-10 04:55:57,073 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:55:57,074 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/98553a9fb8864399a653ee862699ce58 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/98553a9fb8864399a653ee862699ce58 2023-11-10 04:55:57,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/59b8f650e642410e93116f55a4a044ad to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/59b8f650e642410e93116f55a4a044ad 2023-11-10 04:55:57,076 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ce917c69fc9d4ca9a5646ae8059f4768_SeqId_46_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ce917c69fc9d4ca9a5646ae8059f4768_SeqId_46_ 2023-11-10 04:55:57,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ecaa0faafd1d42efb03c910f2ac888e5_SeqId_49_ to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/family1/ecaa0faafd1d42efb03c910f2ac888e5_SeqId_49_ 2023-11-10 04:55:57,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-10 04:55:57,375 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:57,375 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing bc8052c221bbbcbdf08850f6e381c528, disabling compactions & flushes 2023-11-10 04:55:57,375 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:55:57,375 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:55:57,375 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. after waiting 0 ms 2023-11-10 04:55:57,375 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:55:57,375 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:55:57,375 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:55:57,378 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:57,378 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 8b858392c9cfe904206ed137d6282aa0, disabling compactions & flushes 2023-11-10 04:55:57,378 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:55:57,378 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:55:57,378 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. after waiting 0 ms 2023-11-10 04:55:57,378 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:55:57,378 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:55:57,378 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:55:57,380 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-11-10 04:55:57,381 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592157381"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592157381"}]},"ts":"1699592157381"} 2023-11-10 04:55:57,381 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592157381"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592157381"}]},"ts":"1699592157381"} 2023-11-10 04:55:57,381 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1699592157381"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1699592157381"}]},"ts":"1699592157381"} 2023-11-10 04:55:57,383 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-10 04:55:57,384 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-10 04:55:57,384 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592157384"}]},"ts":"1699592157384"} 2023-11-10 04:55:57,385 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-11-10 04:55:57,388 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=bc8052c221bbbcbdf08850f6e381c528, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8b858392c9cfe904206ed137d6282aa0, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f3cbc57e6f6fc77d5d5aa5038d824901, ASSIGN}] 2023-11-10 04:55:57,390 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8b858392c9cfe904206ed137d6282aa0, ASSIGN 2023-11-10 04:55:57,390 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=bc8052c221bbbcbdf08850f6e381c528, ASSIGN 2023-11-10 04:55:57,390 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f3cbc57e6f6fc77d5d5aa5038d824901, ASSIGN 2023-11-10 04:55:57,390 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8b858392c9cfe904206ed137d6282aa0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:57,390 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=bc8052c221bbbcbdf08850f6e381c528, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:57,391 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f3cbc57e6f6fc77d5d5aa5038d824901, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36419,1699592002485; forceNewPlan=false, retain=false 2023-11-10 04:55:57,542 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=f3cbc57e6f6fc77d5d5aa5038d824901, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:57,542 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=8b858392c9cfe904206ed137d6282aa0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:57,542 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=bc8052c221bbbcbdf08850f6e381c528, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:57,542 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592157542"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592157542"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592157542"}]},"ts":"1699592157542"} 2023-11-10 04:55:57,542 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592157542"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592157542"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592157542"}]},"ts":"1699592157542"} 2023-11-10 04:55:57,542 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1699592157542"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1699592157542"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1699592157542"}]},"ts":"1699592157542"} 2023-11-10 04:55:57,544 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure bc8052c221bbbcbdf08850f6e381c528, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:57,544 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=121, ppid=119, state=RUNNABLE; OpenRegionProcedure f3cbc57e6f6fc77d5d5aa5038d824901, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:57,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure 8b858392c9cfe904206ed137d6282aa0, server=jenkins-hbase4.apache.org,36419,1699592002485}] 2023-11-10 04:55:57,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-10 04:55:57,695 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:57,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:55:57,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f3cbc57e6f6fc77d5d5aa5038d824901, NAME => 'testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901.', STARTKEY => '2', ENDKEY => ''} 2023-11-10 04:55:57,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:57,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:57,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:57,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:57,720 INFO [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:57,721 DEBUG [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1 2023-11-10 04:55:57,721 DEBUG [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1 2023-11-10 04:55:57,722 INFO [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3cbc57e6f6fc77d5d5aa5038d824901 columnFamilyName family1 2023-11-10 04:55:57,723 INFO [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] regionserver.HStore(310): Store=f3cbc57e6f6fc77d5d5aa5038d824901/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:57,723 INFO [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:57,724 DEBUG [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2 2023-11-10 04:55:57,724 DEBUG [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2 2023-11-10 04:55:57,724 INFO [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3cbc57e6f6fc77d5d5aa5038d824901 columnFamilyName family2 2023-11-10 04:55:57,725 INFO [StoreOpener-f3cbc57e6f6fc77d5d5aa5038d824901-1] regionserver.HStore(310): Store=f3cbc57e6f6fc77d5d5aa5038d824901/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:57,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:57,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:57,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:57,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:55:57,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:57,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f3cbc57e6f6fc77d5d5aa5038d824901; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72009810, jitterRate=0.0730297863483429}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:57,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:55:57,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901., pid=121, masterSystemTime=1699592157695 2023-11-10 04:55:57,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:55:57,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:55:57,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:55:57,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8b858392c9cfe904206ed137d6282aa0, NAME => 'testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0.', STARTKEY => '1', ENDKEY => '2'} 2023-11-10 04:55:57,732 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=f3cbc57e6f6fc77d5d5aa5038d824901, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:57,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:57,732 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592157731"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592157731"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592157731"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592157731"}]},"ts":"1699592157731"} 2023-11-10 04:55:57,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:57,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:57,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:57,734 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=121, resume processing ppid=119 2023-11-10 04:55:57,734 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=121, ppid=119, state=SUCCESS; OpenRegionProcedure f3cbc57e6f6fc77d5d5aa5038d824901, server=jenkins-hbase4.apache.org,36419,1699592002485 in 189 msec 2023-11-10 04:55:57,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=f3cbc57e6f6fc77d5d5aa5038d824901, ASSIGN in 346 msec 2023-11-10 04:55:57,751 INFO [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:57,752 DEBUG [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1 2023-11-10 04:55:57,753 DEBUG [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1 2023-11-10 04:55:57,753 INFO [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8b858392c9cfe904206ed137d6282aa0 columnFamilyName family1 2023-11-10 04:55:57,753 INFO [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] regionserver.HStore(310): Store=8b858392c9cfe904206ed137d6282aa0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:57,754 INFO [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:57,755 DEBUG [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2 2023-11-10 04:55:57,755 DEBUG [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2 2023-11-10 04:55:57,755 INFO [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8b858392c9cfe904206ed137d6282aa0 columnFamilyName family2 2023-11-10 04:55:57,755 INFO [StoreOpener-8b858392c9cfe904206ed137d6282aa0-1] regionserver.HStore(310): Store=8b858392c9cfe904206ed137d6282aa0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:57,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:57,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:57,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:57,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:55:57,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:57,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8b858392c9cfe904206ed137d6282aa0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65384249, jitterRate=-0.025698766112327576}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:57,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:55:57,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0., pid=122, masterSystemTime=1699592157695 2023-11-10 04:55:57,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:55:57,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:55:57,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:55:57,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bc8052c221bbbcbdf08850f6e381c528, NAME => 'testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528.', STARTKEY => '', ENDKEY => '1'} 2023-11-10 04:55:57,762 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=8b858392c9cfe904206ed137d6282aa0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:57,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:57,762 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1699592157762"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592157762"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592157762"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592157762"}]},"ts":"1699592157762"} 2023-11-10 04:55:57,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-10 04:55:57,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:57,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:57,765 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=122, resume processing ppid=118 2023-11-10 04:55:57,765 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure 8b858392c9cfe904206ed137d6282aa0, server=jenkins-hbase4.apache.org,36419,1699592002485 in 219 msec 2023-11-10 04:55:57,766 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8b858392c9cfe904206ed137d6282aa0, ASSIGN in 377 msec 2023-11-10 04:55:57,783 INFO [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:57,784 DEBUG [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1 2023-11-10 04:55:57,784 DEBUG [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1 2023-11-10 04:55:57,784 INFO [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bc8052c221bbbcbdf08850f6e381c528 columnFamilyName family1 2023-11-10 04:55:57,785 INFO [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] regionserver.HStore(310): Store=bc8052c221bbbcbdf08850f6e381c528/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:57,786 INFO [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:57,787 DEBUG [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2 2023-11-10 04:55:57,787 DEBUG [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2 2023-11-10 04:55:57,787 INFO [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bc8052c221bbbcbdf08850f6e381c528 columnFamilyName family2 2023-11-10 04:55:57,788 INFO [StoreOpener-bc8052c221bbbcbdf08850f6e381c528-1] regionserver.HStore(310): Store=bc8052c221bbbcbdf08850f6e381c528/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-10 04:55:57,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:57,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:57,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-10 04:55:57,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:55:57,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-10 04:55:57,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bc8052c221bbbcbdf08850f6e381c528; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74669393, jitterRate=0.11266066133975983}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-10 04:55:57,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:55:57,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528., pid=120, masterSystemTime=1699592157695 2023-11-10 04:55:57,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:55:57,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:55:57,796 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=bc8052c221bbbcbdf08850f6e381c528, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:57,796 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1699592157796"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1699592157796"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1699592157796"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1699592157796"}]},"ts":"1699592157796"} 2023-11-10 04:55:57,798 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=120, resume processing ppid=117 2023-11-10 04:55:57,798 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure bc8052c221bbbcbdf08850f6e381c528, server=jenkins-hbase4.apache.org,36419,1699592002485 in 253 msec 2023-11-10 04:55:57,800 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=117, resume processing ppid=116 2023-11-10 04:55:57,800 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=bc8052c221bbbcbdf08850f6e381c528, ASSIGN in 410 msec 2023-11-10 04:55:57,800 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-10 04:55:57,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1699592157800"}]},"ts":"1699592157800"} 2023-11-10 04:55:57,801 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-11-10 04:55:57,804 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-11-10 04:55:57,805 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2620 sec 2023-11-10 04:55:58,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-10 04:55:58,650 INFO [Listener at localhost/35765] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-11-10 04:55:58,656 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:58,656 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:58,656 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:55:58,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:55:58,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-10 04:55:58,660 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-10 04:55:58,660 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:58,660 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-10 04:55:58,660 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-10 04:55:58,661 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:55:58,661 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:55:58,662 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:55:58,662 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:58,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:55:58,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:55:58,663 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:55:58,663 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:55:58,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:55:58,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:55:58,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:55:58,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:55:58,664 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-10 04:55:58,664 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-10 04:55:58,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-10 04:55:58,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:55:58,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-10 04:55:58,664 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:55:58,664 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. started... 2023-11-10 04:55:58,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:55:58,664 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:55:58,664 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:55:58,665 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:55:58,665 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. started... 2023-11-10 04:55:58,665 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8b858392c9cfe904206ed137d6282aa0 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-10 04:55:58,665 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. started... 2023-11-10 04:55:58,666 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bc8052c221bbbcbdf08850f6e381c528 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-10 04:55:58,666 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3cbc57e6f6fc77d5d5aa5038d824901 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-10 04:55:59,115 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110d82ebaec8b324c65aa2a23881c77b43c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110d82ebaec8b324c65aa2a23881c77b43c 2023-11-10 04:55:59,124 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110206e6f8b071e47b384bf70c6b05ce3b7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110206e6f8b071e47b384bf70c6b05ce3b7 2023-11-10 04:55:59,125 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110a02a0a7884894f18bfa41a9299f9f124 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110a02a0a7884894f18bfa41a9299f9f124 2023-11-10 04:55:59,139 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/574f08386eb3438d942ca6bac565a469 2023-11-10 04:55:59,532 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/2becc2f7bc6f42f394bf28b2c37aa6f8 2023-11-10 04:55:59,541 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/a9ce51fca3f04401b843d92bf69498dc 2023-11-10 04:55:59,565 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311105a9af14e38364217b80aa50ce530428f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202311105a9af14e38364217b80aa50ce530428f 2023-11-10 04:55:59,570 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311104098291ea2db45598c6f193221c99adc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202311104098291ea2db45598c6f193221c99adc 2023-11-10 04:55:59,578 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/ce0bc3ad3b2c4d06a57fc892cb200d97 2023-11-10 04:55:59,583 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/2becc2f7bc6f42f394bf28b2c37aa6f8 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/2becc2f7bc6f42f394bf28b2c37aa6f8 2023-11-10 04:55:59,607 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/2becc2f7bc6f42f394bf28b2c37aa6f8, entries=4, sequenceid=5, filesize=5.5 K 2023-11-10 04:55:59,608 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/ce0bc3ad3b2c4d06a57fc892cb200d97 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/ce0bc3ad3b2c4d06a57fc892cb200d97 2023-11-10 04:55:59,611 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/ce0bc3ad3b2c4d06a57fc892cb200d97, entries=2, sequenceid=5, filesize=5.1 K 2023-11-10 04:55:59,612 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 8b858392c9cfe904206ed137d6282aa0 in 947ms, sequenceid=5, compaction requested=false 2023-11-10 04:55:59,612 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-11-10 04:55:59,613 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:55:59,613 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:55:59,613 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:55:59,981 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/eb5ea1487ea442f7821f02fa64fd915b 2023-11-10 04:55:59,984 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/574f08386eb3438d942ca6bac565a469 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/574f08386eb3438d942ca6bac565a469 2023-11-10 04:55:59,994 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111048c7ad1fec06479a82aa85155259f6b9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023111048c7ad1fec06479a82aa85155259f6b9 2023-11-10 04:56:00,010 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/574f08386eb3438d942ca6bac565a469, entries=4, sequenceid=5, filesize=5.5 K 2023-11-10 04:56:00,011 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/eb5ea1487ea442f7821f02fa64fd915b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/eb5ea1487ea442f7821f02fa64fd915b 2023-11-10 04:56:00,015 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/eb5ea1487ea442f7821f02fa64fd915b, entries=2, sequenceid=5, filesize=5.1 K 2023-11-10 04:56:00,015 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for bc8052c221bbbcbdf08850f6e381c528 in 1349ms, sequenceid=5, compaction requested=false 2023-11-10 04:56:00,016 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:56:00,016 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:00,016 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:56:00,399 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/ee4e76f5914b4ad681cc0d05abe33738 2023-11-10 04:56:00,403 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/a9ce51fca3f04401b843d92bf69498dc as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/a9ce51fca3f04401b843d92bf69498dc 2023-11-10 04:56:00,426 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/a9ce51fca3f04401b843d92bf69498dc, entries=4, sequenceid=5, filesize=5.5 K 2023-11-10 04:56:00,426 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/ee4e76f5914b4ad681cc0d05abe33738 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/ee4e76f5914b4ad681cc0d05abe33738 2023-11-10 04:56:00,430 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/ee4e76f5914b4ad681cc0d05abe33738, entries=2, sequenceid=5, filesize=5.1 K 2023-11-10 04:56:00,430 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f3cbc57e6f6fc77d5d5aa5038d824901 in 1764ms, sequenceid=5, compaction requested=false 2023-11-10 04:56:00,430 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:56:00,430 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:00,430 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:56:00,431 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:56:00,431 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,431 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-10 04:56:00,431 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-10 04:56:00,432 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,432 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:00,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:00,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:00,433 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:00,433 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:56:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:00,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:00,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:00,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-10 04:56:00,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7fc226d[Count = 0] remaining members to acquire global barrier 2023-11-10 04:56:00,434 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-10 04:56:00,434 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:00,435 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:00,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:00,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:00,436 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,436 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:56:00,436 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-10 04:56:00,436 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-10 04:56:00,436 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:56:00,437 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-10 04:56:00,437 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,437 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:00,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:00,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:00,437 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-10 04:56:00,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:00,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:00,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:00,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:00,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:00,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:56:00,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-10 04:56:00,440 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-10 04:56:00,440 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:56:00,440 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:56:00,440 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-10 04:56:00,440 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:56:00,451 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:00,451 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:00,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:00,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:00,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:00,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:00,451 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:00,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:00,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:00,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:00,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:00,451 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:00,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:00,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:00,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:00,452 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:00,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:00,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:56:00,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:00,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:56:00,455 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-10 04:56:00,455 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:56:00,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:56:00,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-10 04:56:00,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:00,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:00,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:00,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:00,455 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-10 04:56:00,455 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:56:00,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:00,455 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:00,456 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:00,456 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:00,556 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:56:00,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:56:00,557 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:56:00,558 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:56:00,558 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:56:00,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:56:00,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-10 04:56:00,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-10 04:56:00,562 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-10 04:56:00,562 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:56:00,562 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-10 04:56:00,562 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-10 04:56:00,562 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:00,562 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:00,564 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:56:00,564 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:56:00,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:56:00,564 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:00,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:00,564 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:56:00,564 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:00,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:56:00,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:00,565 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-10 04:56:00,565 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-10 04:56:00,565 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-10 04:56:00,565 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-10 04:56:00,565 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:56:00,565 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-10 04:56:00,565 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:00,565 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:00,565 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:00,565 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. started... 2023-11-10 04:56:00,565 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. started... 2023-11-10 04:56:00,565 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:56:00,566 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. started... 2023-11-10 04:56:00,566 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8b858392c9cfe904206ed137d6282aa0 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-10 04:56:00,566 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bc8052c221bbbcbdf08850f6e381c528 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-10 04:56:00,566 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:56:00,566 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3cbc57e6f6fc77d5d5aa5038d824901 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-10 04:56:00,606 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110def95f5928c745e98b81bf0448fc8f8d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110def95f5928c745e98b81bf0448fc8f8d 2023-11-10 04:56:01,008 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110e766efefe0044f27b44594d81bd3fb24 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110e766efefe0044f27b44594d81bd3fb24 2023-11-10 04:56:01,018 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110289a29f93eaa416aaac3ad05f66e9770 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110289a29f93eaa416aaac3ad05f66e9770 2023-11-10 04:56:01,020 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/c8632ace49d442a98c8747ad89d96467 2023-11-10 04:56:01,026 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/689c30cbf48a44f3b85d300a0983e85f 2023-11-10 04:56:01,440 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/df8a092643334a929798b364a7ec0559 2023-11-10 04:56:01,458 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111002f6cb14e9b34de3a1fa0aad6d37c06e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023111002f6cb14e9b34de3a1fa0aad6d37c06e 2023-11-10 04:56:01,464 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110cf28754618094220a11f043da0caa267 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231110cf28754618094220a11f043da0caa267 2023-11-10 04:56:01,863 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/487c4d5b5c0a4345af1cf5cbd398d8ed 2023-11-10 04:56:01,867 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/c8632ace49d442a98c8747ad89d96467 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/c8632ace49d442a98c8747ad89d96467 2023-11-10 04:56:01,868 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/09c0f4332bf342c4b25885e5dd63fa93 2023-11-10 04:56:01,873 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/689c30cbf48a44f3b85d300a0983e85f as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/689c30cbf48a44f3b85d300a0983e85f 2023-11-10 04:56:01,886 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110707e753adc0f4664a21359fe5c0d233c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231110707e753adc0f4664a21359fe5c0d233c 2023-11-10 04:56:01,902 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/c8632ace49d442a98c8747ad89d96467, entries=4, sequenceid=9, filesize=5.5 K 2023-11-10 04:56:01,903 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/487c4d5b5c0a4345af1cf5cbd398d8ed as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/487c4d5b5c0a4345af1cf5cbd398d8ed 2023-11-10 04:56:01,903 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/689c30cbf48a44f3b85d300a0983e85f, entries=4, sequenceid=9, filesize=5.5 K 2023-11-10 04:56:01,904 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/09c0f4332bf342c4b25885e5dd63fa93 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/09c0f4332bf342c4b25885e5dd63fa93 2023-11-10 04:56:01,907 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/487c4d5b5c0a4345af1cf5cbd398d8ed, entries=2, sequenceid=9, filesize=5.1 K 2023-11-10 04:56:01,907 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/09c0f4332bf342c4b25885e5dd63fa93, entries=2, sequenceid=9, filesize=5.1 K 2023-11-10 04:56:01,908 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f3cbc57e6f6fc77d5d5aa5038d824901 in 1342ms, sequenceid=9, compaction requested=false 2023-11-10 04:56:01,908 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 8b858392c9cfe904206ed137d6282aa0 in 1342ms, sequenceid=9, compaction requested=false 2023-11-10 04:56:01,908 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:56:01,908 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:56:01,908 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:01,908 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:01,908 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:56:01,908 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:56:02,295 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/3f3cfd25000e435fb3bfa146ba400383 2023-11-10 04:56:02,298 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/df8a092643334a929798b364a7ec0559 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/df8a092643334a929798b364a7ec0559 2023-11-10 04:56:02,325 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/df8a092643334a929798b364a7ec0559, entries=4, sequenceid=9, filesize=5.5 K 2023-11-10 04:56:02,325 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/3f3cfd25000e435fb3bfa146ba400383 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/3f3cfd25000e435fb3bfa146ba400383 2023-11-10 04:56:02,329 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/3f3cfd25000e435fb3bfa146ba400383, entries=2, sequenceid=9, filesize=5.1 K 2023-11-10 04:56:02,330 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for bc8052c221bbbcbdf08850f6e381c528 in 1763ms, sequenceid=9, compaction requested=false 2023-11-10 04:56:02,330 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:56:02,330 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:02,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:56:02,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:56:02,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-10 04:56:02,330 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-10 04:56:02,333 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:02,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:02,333 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:56:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:02,335 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-10 04:56:02,335 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cc20687[Count = 0] remaining members to acquire global barrier 2023-11-10 04:56:02,335 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-10 04:56:02,335 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:02,336 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:02,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:02,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:02,336 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-10 04:56:02,336 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-10 04:56:02,336 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:56:02,336 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,336 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:56:02,338 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,338 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-10 04:56:02,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:02,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:02,338 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:02,338 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-10 04:56:02,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:02,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:02,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:56:02,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-10 04:56:02,340 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-10 04:56:02,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:56:02,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:56:02,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-10 04:56:02,340 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:56:02,342 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:02,342 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:02,342 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:02,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:02,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:02,343 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:02,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:02,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:02,344 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:02,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:02,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:02,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:02,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,348 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:02,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:02,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-10 04:56:02,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:02,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:02,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-10 04:56:02,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-10 04:56:02,348 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:56:02,449 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:56:02,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:56:02,451 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:56:02,451 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:56:02,451 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:56:02,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:56:02,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-10 04:56:02,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-10 04:56:02,455 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-10 04:56:02,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:56:02,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-10 04:56:02,455 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-10 04:56:02,456 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:02,456 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:02,457 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:56:02,457 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:56:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:56:02,458 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:02,458 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:56:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:02,458 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:56:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:02,459 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-10 04:56:02,463 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-10 04:56:02,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-10 04:56:02,463 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-10 04:56:02,463 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:56:02,463 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-10 04:56:02,463 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:02,463 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:02,463 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:02,463 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. started... 2023-11-10 04:56:02,463 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. started... 2023-11-10 04:56:02,463 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:56:02,463 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8b858392c9cfe904206ed137d6282aa0 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-10 04:56:02,463 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. started... 2023-11-10 04:56:02,463 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3cbc57e6f6fc77d5d5aa5038d824901 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-10 04:56:02,463 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bc8052c221bbbcbdf08850f6e381c528 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-10 04:56:02,463 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:56:02,518 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110927ec0cd8e2945709d7b833aaebeff38 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110927ec0cd8e2945709d7b833aaebeff38 2023-11-10 04:56:02,745 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:56:02,745 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x239feebf to 127.0.0.1:51169 2023-11-10 04:56:02,745 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:56:02,746 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:56:02,746 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x77e9b0c7 to 127.0.0.1:51169 2023-11-10 04:56:02,746 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:56:02,919 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110a4a3a96209514de1bc3de48d349d1943 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110a4a3a96209514de1bc3de48d349d1943 2023-11-10 04:56:02,931 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110813447ec2cf74d5ba6529ab59cf1d8cf to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110813447ec2cf74d5ba6529ab59cf1d8cf 2023-11-10 04:56:02,935 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/e9b980cbdf4f43618ef6b8f9747ae18e 2023-11-10 04:56:02,951 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/19a5037e0b7a48b0a90f6733b108d9db 2023-11-10 04:56:03,280 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-10 04:56:03,335 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/b3bc9ab5f6ed47eeb4935b9ef153a49a 2023-11-10 04:56:03,378 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311108cd0ab3b615a420a8b608ec53929cc4d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202311108cd0ab3b615a420a8b608ec53929cc4d 2023-11-10 04:56:03,385 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/503b3bbcbf1643f2a9a8161e6e339609 2023-11-10 04:56:03,388 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111034c4a31eff334151a6256f353cad1f32 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023111034c4a31eff334151a6256f353cad1f32 2023-11-10 04:56:03,389 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/e9b980cbdf4f43618ef6b8f9747ae18e as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/e9b980cbdf4f43618ef6b8f9747ae18e 2023-11-10 04:56:03,415 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/e9b980cbdf4f43618ef6b8f9747ae18e, entries=4, sequenceid=13, filesize=5.5 K 2023-11-10 04:56:03,416 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/503b3bbcbf1643f2a9a8161e6e339609 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/503b3bbcbf1643f2a9a8161e6e339609 2023-11-10 04:56:03,419 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/503b3bbcbf1643f2a9a8161e6e339609, entries=2, sequenceid=13, filesize=5.1 K 2023-11-10 04:56:03,420 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for bc8052c221bbbcbdf08850f6e381c528 in 957ms, sequenceid=13, compaction requested=true 2023-11-10 04:56:03,420 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:56:03,420 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:03,420 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:56:03,768 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023111072bb6a78a87b481ab5f8f71e6a0a145b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023111072bb6a78a87b481ab5f8f71e6a0a145b 2023-11-10 04:56:03,801 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/e04091a8d6154fad8235b9898c61a173 2023-11-10 04:56:03,804 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/19a5037e0b7a48b0a90f6733b108d9db as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/19a5037e0b7a48b0a90f6733b108d9db 2023-11-10 04:56:03,827 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/19a5037e0b7a48b0a90f6733b108d9db, entries=4, sequenceid=13, filesize=5.5 K 2023-11-10 04:56:03,828 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/e04091a8d6154fad8235b9898c61a173 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/e04091a8d6154fad8235b9898c61a173 2023-11-10 04:56:03,831 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/e04091a8d6154fad8235b9898c61a173, entries=2, sequenceid=13, filesize=5.1 K 2023-11-10 04:56:03,832 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f3cbc57e6f6fc77d5d5aa5038d824901 in 1369ms, sequenceid=13, compaction requested=true 2023-11-10 04:56:03,832 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:56:03,832 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:03,832 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:56:04,173 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/a3801c7a00ee41e4a313f9f9f05cfdde 2023-11-10 04:56:04,177 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/b3bc9ab5f6ed47eeb4935b9ef153a49a as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/b3bc9ab5f6ed47eeb4935b9ef153a49a 2023-11-10 04:56:04,200 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/b3bc9ab5f6ed47eeb4935b9ef153a49a, entries=4, sequenceid=13, filesize=5.5 K 2023-11-10 04:56:04,200 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/a3801c7a00ee41e4a313f9f9f05cfdde as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a3801c7a00ee41e4a313f9f9f05cfdde 2023-11-10 04:56:04,204 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a3801c7a00ee41e4a313f9f9f05cfdde, entries=2, sequenceid=13, filesize=5.1 K 2023-11-10 04:56:04,204 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 8b858392c9cfe904206ed137d6282aa0 in 1741ms, sequenceid=13, compaction requested=true 2023-11-10 04:56:04,205 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:56:04,205 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:04,205 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:56:04,205 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:56:04,205 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,205 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-10 04:56:04,205 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-10 04:56:04,208 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:04,208 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:04,208 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:04,208 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:56:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:04,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:04,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:04,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:04,209 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-10 04:56:04,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6347fdda[Count = 0] remaining members to acquire global barrier 2023-11-10 04:56:04,210 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-10 04:56:04,210 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:04,211 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:04,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:04,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:04,211 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-10 04:56:04,211 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-10 04:56:04,211 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:56:04,211 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,211 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:56:04,212 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,212 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-10 04:56:04,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,212 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:04,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:04,212 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-10 04:56:04,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:04,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:04,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:56:04,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-10 04:56:04,215 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-10 04:56:04,215 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:56:04,215 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:56:04,215 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-10 04:56:04,215 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:56:04,217 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:04,217 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:04,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:04,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:04,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:04,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:04,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:04,217 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:04,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:04,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:04,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:04,217 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:04,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:04,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:04,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:04,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,219 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:04,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:04,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,222 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:56:04,222 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:56:04,222 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:04,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:56:04,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-10 04:56:04,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:56:04,222 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:04,222 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:04,222 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:04,223 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:04,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:04,223 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:04,223 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-10 04:56:04,223 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:56:04,223 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:04,223 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:04,323 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:56:04,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:56:04,325 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:56:04,325 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:56:04,325 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. with WAL disabled. Data may be lost in the event of a crash. 2023-11-10 04:56:04,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:56:04,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-10 04:56:04,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-10 04:56:04,329 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-10 04:56:04,329 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:56:04,329 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-10 04:56:04,329 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-10 04:56:04,330 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:04,330 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:04,331 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:56:04,331 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:56:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:56:04,332 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:04,332 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:56:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:04,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:56:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:04,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-10 04:56:04,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-10 04:56:04,333 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-10 04:56:04,333 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-10 04:56:04,333 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:56:04,333 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-10 04:56:04,333 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:04,333 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. started... 2023-11-10 04:56:04,333 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:56:04,333 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:56:04,333 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:04,333 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. started... 2023-11-10 04:56:04,333 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:04,333 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3cbc57e6f6fc77d5d5aa5038d824901 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-10 04:56:04,333 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8b858392c9cfe904206ed137d6282aa0 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-10 04:56:04,333 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. started... 2023-11-10 04:56:04,334 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bc8052c221bbbcbdf08850f6e381c528 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-10 04:56:04,379 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110f4c4436d045947c59bc20bc6114bc166 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110f4c4436d045947c59bc20bc6114bc166 2023-11-10 04:56:04,394 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/7eef2883689045afa6cddc0a6abca386 2023-11-10 04:56:04,425 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110267a61dfba7c412ea7d648b0e992666b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231110267a61dfba7c412ea7d648b0e992666b 2023-11-10 04:56:04,429 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/f1e237ffc764469c9473d6784bfddf92 2023-11-10 04:56:04,433 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/7eef2883689045afa6cddc0a6abca386 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/7eef2883689045afa6cddc0a6abca386 2023-11-10 04:56:04,460 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/7eef2883689045afa6cddc0a6abca386, entries=4, sequenceid=17, filesize=5.5 K 2023-11-10 04:56:04,460 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/f1e237ffc764469c9473d6784bfddf92 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/f1e237ffc764469c9473d6784bfddf92 2023-11-10 04:56:04,464 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/f1e237ffc764469c9473d6784bfddf92, entries=2, sequenceid=17, filesize=5.1 K 2023-11-10 04:56:04,465 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for bc8052c221bbbcbdf08850f6e381c528 in 131ms, sequenceid=17, compaction requested=true 2023-11-10 04:56:04,465 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:56:04,465 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:04,465 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:56:04,784 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111084f139d451a64ad0a821d43d5623cbf3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023111084f139d451a64ad0a821d43d5623cbf3 2023-11-10 04:56:04,794 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110edd66af8ef84455383303ac73a797f2b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110edd66af8ef84455383303ac73a797f2b 2023-11-10 04:56:05,198 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/63f4b67db25f475394530846b98731f5 2023-11-10 04:56:05,215 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/220658e54eed40b6abff719a6b079d36 2023-11-10 04:56:05,633 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023111074f69fe4f70642a0a5059eed327ae5bb to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023111074f69fe4f70642a0a5059eed327ae5bb 2023-11-10 04:56:05,649 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110fe61fbce16394425891766783092c8e1 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231110fe61fbce16394425891766783092c8e1 2023-11-10 04:56:06,038 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/d89a000696c6452b9618a0e28824f2d7 2023-11-10 04:56:06,042 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/63f4b67db25f475394530846b98731f5 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/63f4b67db25f475394530846b98731f5 2023-11-10 04:56:06,053 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/a54b3513730541a9b316f9f2fdc6fd03 2023-11-10 04:56:06,057 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/220658e54eed40b6abff719a6b079d36 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/220658e54eed40b6abff719a6b079d36 2023-11-10 04:56:06,065 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/63f4b67db25f475394530846b98731f5, entries=4, sequenceid=17, filesize=5.5 K 2023-11-10 04:56:06,066 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/d89a000696c6452b9618a0e28824f2d7 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/d89a000696c6452b9618a0e28824f2d7 2023-11-10 04:56:06,069 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/d89a000696c6452b9618a0e28824f2d7, entries=2, sequenceid=17, filesize=5.1 K 2023-11-10 04:56:06,070 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f3cbc57e6f6fc77d5d5aa5038d824901 in 1737ms, sequenceid=17, compaction requested=true 2023-11-10 04:56:06,070 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:56:06,070 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:06,070 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:56:06,079 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/220658e54eed40b6abff719a6b079d36, entries=4, sequenceid=17, filesize=5.5 K 2023-11-10 04:56:06,079 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/a54b3513730541a9b316f9f2fdc6fd03 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a54b3513730541a9b316f9f2fdc6fd03 2023-11-10 04:56:06,083 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a54b3513730541a9b316f9f2fdc6fd03, entries=2, sequenceid=17, filesize=5.1 K 2023-11-10 04:56:06,084 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 8b858392c9cfe904206ed137d6282aa0 in 1751ms, sequenceid=17, compaction requested=true 2023-11-10 04:56:06,084 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:56:06,084 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-10 04:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-10 04:56:06,086 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,086 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:06,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:06,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:06,086 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:06,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:06,086 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:56:06,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:06,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:06,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:06,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-10 04:56:06,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e7d170[Count = 0] remaining members to acquire global barrier 2023-11-10 04:56:06,087 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-10 04:56:06,088 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:06,093 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:06,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:06,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:06,093 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-10 04:56:06,093 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-10 04:56:06,093 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:56:06,093 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,093 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:56:06,095 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,095 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-10 04:56:06,095 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:06,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:06,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:06,095 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-10 04:56:06,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:06,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:06,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:06,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:06,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:06,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:56:06,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-10 04:56:06,097 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-10 04:56:06,097 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:56:06,097 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:56:06,097 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-10 04:56:06,097 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:56:06,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:06,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:06,099 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:06,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:06,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:06,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:06,099 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:06,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:06,101 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:06,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:06,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:06,104 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:06,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-10 04:56:06,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-10 04:56:06,104 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:56:06,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:06,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:06,205 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:56:06,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:56:06,214 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:56:06,221 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:56:06,228 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:56:06,229 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:56:06,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-10 04:56:06,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-10 04:56:06,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-10 04:56:06,337 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-10 04:56:06,337 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:56:06,337 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-10 04:56:06,337 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-10 04:56:06,338 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:06,338 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:06,339 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:56:06,339 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:56:06,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:56:06,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:06,340 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-10 04:56:06,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:06,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:06,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-10 04:56:06,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:06,341 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-10 04:56:06,341 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-10 04:56:06,341 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-10 04:56:06,341 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-10 04:56:06,341 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-10 04:56:06,341 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-10 04:56:06,341 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:06,341 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-10 04:56:06,341 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-10 04:56:06,341 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:06,341 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:06,342 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. started... 2023-11-10 04:56:06,341 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. started... 2023-11-10 04:56:06,342 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing f3cbc57e6f6fc77d5d5aa5038d824901 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-10 04:56:06,342 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. started... 2023-11-10 04:56:06,342 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8b858392c9cfe904206ed137d6282aa0 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-10 04:56:06,342 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing bc8052c221bbbcbdf08850f6e381c528 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-10 04:56:06,782 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/05e8576a08d147ae8453929772103fe4 2023-11-10 04:56:06,791 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/1b4442c6b4f74fbeb67d25af2a52f77c 2023-11-10 04:56:06,791 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/f6f43ab3d1504e7584075b11ac59fe69 2023-11-10 04:56:06,806 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05e8576a08d147ae8453929772103fe4 2023-11-10 04:56:06,813 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1b4442c6b4f74fbeb67d25af2a52f77c 2023-11-10 04:56:06,825 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f6f43ab3d1504e7584075b11ac59fe69 2023-11-10 04:56:07,621 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/fb921f44afb74539bcbf45d7ded1389d 2023-11-10 04:56:07,624 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb921f44afb74539bcbf45d7ded1389d 2023-11-10 04:56:07,624 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/05e8576a08d147ae8453929772103fe4 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/05e8576a08d147ae8453929772103fe4 2023-11-10 04:56:07,629 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/cf84e91255c0479a948fc94992476372 2023-11-10 04:56:07,632 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cf84e91255c0479a948fc94992476372 2023-11-10 04:56:07,632 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/1b4442c6b4f74fbeb67d25af2a52f77c as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/1b4442c6b4f74fbeb67d25af2a52f77c 2023-11-10 04:56:07,635 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/322f97ed9cb1410aa18c4b2c652e7a51 2023-11-10 04:56:07,638 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 322f97ed9cb1410aa18c4b2c652e7a51 2023-11-10 04:56:07,638 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/f6f43ab3d1504e7584075b11ac59fe69 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/f6f43ab3d1504e7584075b11ac59fe69 2023-11-10 04:56:07,648 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05e8576a08d147ae8453929772103fe4 2023-11-10 04:56:07,648 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/05e8576a08d147ae8453929772103fe4, entries=3, sequenceid=23, filesize=5.2 K 2023-11-10 04:56:07,649 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/fb921f44afb74539bcbf45d7ded1389d as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/fb921f44afb74539bcbf45d7ded1389d 2023-11-10 04:56:07,655 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb921f44afb74539bcbf45d7ded1389d 2023-11-10 04:56:07,655 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/fb921f44afb74539bcbf45d7ded1389d, entries=1, sequenceid=23, filesize=5.0 K 2023-11-10 04:56:07,656 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f3cbc57e6f6fc77d5d5aa5038d824901 in 1314ms, sequenceid=23, compaction requested=true 2023-11-10 04:56:07,656 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:56:07,656 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:07,656 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-10 04:56:07,656 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1b4442c6b4f74fbeb67d25af2a52f77c 2023-11-10 04:56:07,656 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/1b4442c6b4f74fbeb67d25af2a52f77c, entries=3, sequenceid=23, filesize=5.2 K 2023-11-10 04:56:07,657 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/cf84e91255c0479a948fc94992476372 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/cf84e91255c0479a948fc94992476372 2023-11-10 04:56:07,661 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cf84e91255c0479a948fc94992476372 2023-11-10 04:56:07,661 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/cf84e91255c0479a948fc94992476372, entries=1, sequenceid=23, filesize=5.0 K 2023-11-10 04:56:07,661 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for bc8052c221bbbcbdf08850f6e381c528 in 1319ms, sequenceid=23, compaction requested=true 2023-11-10 04:56:07,661 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:56:07,662 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:07,662 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-10 04:56:07,670 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f6f43ab3d1504e7584075b11ac59fe69 2023-11-10 04:56:07,670 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/f6f43ab3d1504e7584075b11ac59fe69, entries=3, sequenceid=23, filesize=5.2 K 2023-11-10 04:56:07,671 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/322f97ed9cb1410aa18c4b2c652e7a51 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/322f97ed9cb1410aa18c4b2c652e7a51 2023-11-10 04:56:07,674 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 322f97ed9cb1410aa18c4b2c652e7a51 2023-11-10 04:56:07,674 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/322f97ed9cb1410aa18c4b2c652e7a51, entries=1, sequenceid=23, filesize=5.0 K 2023-11-10 04:56:07,675 INFO [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 8b858392c9cfe904206ed137d6282aa0 in 1333ms, sequenceid=23, compaction requested=true 2023-11-10 04:56:07,675 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:56:07,675 DEBUG [rs(jenkins-hbase4.apache.org,36419,1699592002485)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:07,675 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-10 04:56:07,675 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-10 04:56:07,675 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,675 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-10 04:56:07,675 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-10 04:56:07,677 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,677 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:07,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:07,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:07,677 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:07,677 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-10 04:56:07,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:07,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:07,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:07,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:07,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36419,1699592002485' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-10 04:56:07,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@606e648b[Count = 0] remaining members to acquire global barrier 2023-11-10 04:56:07,679 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-10 04:56:07,679 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:07,680 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:07,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:07,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:07,680 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-10 04:56:07,680 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-10 04:56:07,680 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36419,1699592002485' in zk 2023-11-10 04:56:07,680 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,680 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-10 04:56:07,683 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-10 04:56:07,683 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,683 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:07,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:07,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:07,683 DEBUG [member: 'jenkins-hbase4.apache.org,36419,1699592002485' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-10 04:56:07,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:07,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:07,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:07,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:07,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:07,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36419,1699592002485': 2023-11-10 04:56:07,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36419,1699592002485' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-10 04:56:07,685 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-10 04:56:07,685 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-10 04:56:07,685 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-10 04:56:07,685 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-10 04:56:07,685 INFO [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-10 04:56:07,687 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:07,687 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:07,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:07,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-10 04:56:07,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-10 04:56:07,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:07,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:07,687 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:07,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:07,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:07,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-10 04:56:07,688 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:07,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:07,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-10 04:56:07,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:07,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,689 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-10 04:56:07,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-10 04:56:07,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,691 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,691 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-10 04:56:07,691 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:07,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-10 04:56:07,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-10 04:56:07,691 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-10 04:56:07,691 DEBUG [(jenkins-hbase4.apache.org,46447,1699592001315)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-10 04:56:07,691 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-10 04:56:07,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-10 04:56:07,692 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:07,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-10 04:56:07,692 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:07,692 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-10 04:56:07,692 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-10 04:56:07,692 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-10 04:56:07,692 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-10 04:56:07,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-10 04:56:07,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-10 04:56:07,792 DEBUG [Listener at localhost/35765] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-10 04:56:07,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46447] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-10 04:56:07,793 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:56:07,793 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): 8b858392c9cfe904206ed137d6282aa0/family1 is initiating major compaction (all files) 2023-11-10 04:56:07,794 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:56:07,794 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:56:07,794 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of 8b858392c9cfe904206ed137d6282aa0/family1 in testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:07,794 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/2becc2f7bc6f42f394bf28b2c37aa6f8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/689c30cbf48a44f3b85d300a0983e85f, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/b3bc9ab5f6ed47eeb4935b9ef153a49a, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/220658e54eed40b6abff719a6b079d36, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/f6f43ab3d1504e7584075b11ac59fe69] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp, totalSize=27.1 K 2023-11-10 04:56:07,794 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 2becc2f7bc6f42f394bf28b2c37aa6f8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592158656 2023-11-10 04:56:07,795 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 689c30cbf48a44f3b85d300a0983e85f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592160558 2023-11-10 04:56:07,795 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting b3bc9ab5f6ed47eeb4935b9ef153a49a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592162451 2023-11-10 04:56:07,795 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 220658e54eed40b6abff719a6b079d36, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592164325 2023-11-10 04:56:07,796 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting f6f43ab3d1504e7584075b11ac59fe69, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:56:07,804 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): 8b858392c9cfe904206ed137d6282aa0#family1#compaction#197 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:56:08,255 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231110c4035e1acb844a3e86813278d66aa446_del 2023-11-10 04:56:08,255 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110c4035e1acb844a3e86813278d66aa446_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110c4035e1acb844a3e86813278d66aa446_del 2023-11-10 04:56:09,106 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f343d6d160d646fabcb36df32bbff5c4 2023-11-10 04:56:09,107 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family1/f343d6d160d646fabcb36df32bbff5c4 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/f343d6d160d646fabcb36df32bbff5c4 2023-11-10 04:56:09,128 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f343d6d160d646fabcb36df32bbff5c4 2023-11-10 04:56:09,129 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8b858392c9cfe904206ed137d6282aa0/family1 of 8b858392c9cfe904206ed137d6282aa0 into f343d6d160d646fabcb36df32bbff5c4(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:56:09,129 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:56:09,129 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:56:09,130 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): 8b858392c9cfe904206ed137d6282aa0/family2 is initiating major compaction (all files) 2023-11-10 04:56:09,130 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:56:09,130 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:56:09,130 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of 8b858392c9cfe904206ed137d6282aa0/family2 in testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:09,130 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/ce0bc3ad3b2c4d06a57fc892cb200d97, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/09c0f4332bf342c4b25885e5dd63fa93, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a3801c7a00ee41e4a313f9f9f05cfdde, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a54b3513730541a9b316f9f2fdc6fd03, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/322f97ed9cb1410aa18c4b2c652e7a51] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp, totalSize=25.4 K 2023-11-10 04:56:09,130 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting ce0bc3ad3b2c4d06a57fc892cb200d97, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592158656 2023-11-10 04:56:09,131 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 09c0f4332bf342c4b25885e5dd63fa93, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592160558 2023-11-10 04:56:09,131 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting a3801c7a00ee41e4a313f9f9f05cfdde, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592162451 2023-11-10 04:56:09,131 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting a54b3513730541a9b316f9f2fdc6fd03, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592164325 2023-11-10 04:56:09,132 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 322f97ed9cb1410aa18c4b2c652e7a51, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:56:09,139 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): 8b858392c9cfe904206ed137d6282aa0#family2#compaction#198 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:56:09,172 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/2becc2f7bc6f42f394bf28b2c37aa6f8, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/689c30cbf48a44f3b85d300a0983e85f, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/b3bc9ab5f6ed47eeb4935b9ef153a49a, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/220658e54eed40b6abff719a6b079d36, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/f6f43ab3d1504e7584075b11ac59fe69] to archive 2023-11-10 04:56:09,172 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:09,174 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/2becc2f7bc6f42f394bf28b2c37aa6f8 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/2becc2f7bc6f42f394bf28b2c37aa6f8 2023-11-10 04:56:09,175 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/689c30cbf48a44f3b85d300a0983e85f to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/689c30cbf48a44f3b85d300a0983e85f 2023-11-10 04:56:09,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/b3bc9ab5f6ed47eeb4935b9ef153a49a to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/b3bc9ab5f6ed47eeb4935b9ef153a49a 2023-11-10 04:56:09,177 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/220658e54eed40b6abff719a6b079d36 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/220658e54eed40b6abff719a6b079d36 2023-11-10 04:56:09,178 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/f6f43ab3d1504e7584075b11ac59fe69 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/f6f43ab3d1504e7584075b11ac59fe69 2023-11-10 04:56:09,546 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231110b59fdf5543344924b8ab1f699f4d295b_del 2023-11-10 04:56:09,547 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231110b59fdf5543344924b8ab1f699f4d295b_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231110b59fdf5543344924b8ab1f699f4d295b_del 2023-11-10 04:56:10,358 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3765fd7e71af42dba5227f04d37d1781 2023-11-10 04:56:10,359 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/.tmp/family2/3765fd7e71af42dba5227f04d37d1781 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/3765fd7e71af42dba5227f04d37d1781 2023-11-10 04:56:10,362 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3765fd7e71af42dba5227f04d37d1781 2023-11-10 04:56:10,363 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8b858392c9cfe904206ed137d6282aa0/family2 of 8b858392c9cfe904206ed137d6282aa0 into 3765fd7e71af42dba5227f04d37d1781(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:56:10,363 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:56:10,363 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:56:10,363 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): bc8052c221bbbcbdf08850f6e381c528/family1 is initiating major compaction (all files) 2023-11-10 04:56:10,364 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:56:10,364 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:56:10,365 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of bc8052c221bbbcbdf08850f6e381c528/family1 in testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:10,366 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/574f08386eb3438d942ca6bac565a469, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/df8a092643334a929798b364a7ec0559, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/e9b980cbdf4f43618ef6b8f9747ae18e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/7eef2883689045afa6cddc0a6abca386, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/1b4442c6b4f74fbeb67d25af2a52f77c] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp, totalSize=27.1 K 2023-11-10 04:56:10,366 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 574f08386eb3438d942ca6bac565a469, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592158656 2023-11-10 04:56:10,366 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting df8a092643334a929798b364a7ec0559, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592160557 2023-11-10 04:56:10,367 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting e9b980cbdf4f43618ef6b8f9747ae18e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592162451 2023-11-10 04:56:10,367 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 7eef2883689045afa6cddc0a6abca386, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592164325 2023-11-10 04:56:10,367 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 1b4442c6b4f74fbeb67d25af2a52f77c, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:56:10,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/ce0bc3ad3b2c4d06a57fc892cb200d97, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/09c0f4332bf342c4b25885e5dd63fa93, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a3801c7a00ee41e4a313f9f9f05cfdde, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a54b3513730541a9b316f9f2fdc6fd03, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/322f97ed9cb1410aa18c4b2c652e7a51] to archive 2023-11-10 04:56:10,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:10,374 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/ce0bc3ad3b2c4d06a57fc892cb200d97 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/ce0bc3ad3b2c4d06a57fc892cb200d97 2023-11-10 04:56:10,375 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): bc8052c221bbbcbdf08850f6e381c528#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:56:10,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/09c0f4332bf342c4b25885e5dd63fa93 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/09c0f4332bf342c4b25885e5dd63fa93 2023-11-10 04:56:10,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a3801c7a00ee41e4a313f9f9f05cfdde to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a3801c7a00ee41e4a313f9f9f05cfdde 2023-11-10 04:56:10,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a54b3513730541a9b316f9f2fdc6fd03 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/a54b3513730541a9b316f9f2fdc6fd03 2023-11-10 04:56:10,379 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/322f97ed9cb1410aa18c4b2c652e7a51 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family2/322f97ed9cb1410aa18c4b2c652e7a51 2023-11-10 04:56:10,819 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231110f64456ff97ce4b2f98a78bc56bb5fdb1_del 2023-11-10 04:56:10,819 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231110f64456ff97ce4b2f98a78bc56bb5fdb1_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110f64456ff97ce4b2f98a78bc56bb5fdb1_del 2023-11-10 04:56:11,270 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b30a1d0b64e141d7862419f24899a82e 2023-11-10 04:56:11,270 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family1/b30a1d0b64e141d7862419f24899a82e as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/b30a1d0b64e141d7862419f24899a82e 2023-11-10 04:56:11,292 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b30a1d0b64e141d7862419f24899a82e 2023-11-10 04:56:11,292 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bc8052c221bbbcbdf08850f6e381c528/family1 of bc8052c221bbbcbdf08850f6e381c528 into b30a1d0b64e141d7862419f24899a82e(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-10 04:56:11,293 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:56:11,293 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:56:11,293 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): bc8052c221bbbcbdf08850f6e381c528/family2 is initiating major compaction (all files) 2023-11-10 04:56:11,293 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:56:11,293 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:56:11,293 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of bc8052c221bbbcbdf08850f6e381c528/family2 in testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:11,293 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/eb5ea1487ea442f7821f02fa64fd915b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/3f3cfd25000e435fb3bfa146ba400383, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/503b3bbcbf1643f2a9a8161e6e339609, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/f1e237ffc764469c9473d6784bfddf92, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/cf84e91255c0479a948fc94992476372] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp, totalSize=25.4 K 2023-11-10 04:56:11,294 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting eb5ea1487ea442f7821f02fa64fd915b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592158656 2023-11-10 04:56:11,294 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 3f3cfd25000e435fb3bfa146ba400383, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592160557 2023-11-10 04:56:11,295 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 503b3bbcbf1643f2a9a8161e6e339609, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592162451 2023-11-10 04:56:11,295 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting f1e237ffc764469c9473d6784bfddf92, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592164325 2023-11-10 04:56:11,295 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting cf84e91255c0479a948fc94992476372, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:56:11,303 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): bc8052c221bbbcbdf08850f6e381c528#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:56:11,379 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/574f08386eb3438d942ca6bac565a469, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/df8a092643334a929798b364a7ec0559, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/e9b980cbdf4f43618ef6b8f9747ae18e, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/7eef2883689045afa6cddc0a6abca386, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/1b4442c6b4f74fbeb67d25af2a52f77c] to archive 2023-11-10 04:56:11,380 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:11,382 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/574f08386eb3438d942ca6bac565a469 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/574f08386eb3438d942ca6bac565a469 2023-11-10 04:56:11,383 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/df8a092643334a929798b364a7ec0559 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/df8a092643334a929798b364a7ec0559 2023-11-10 04:56:11,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/e9b980cbdf4f43618ef6b8f9747ae18e to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/e9b980cbdf4f43618ef6b8f9747ae18e 2023-11-10 04:56:11,385 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/7eef2883689045afa6cddc0a6abca386 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/7eef2883689045afa6cddc0a6abca386 2023-11-10 04:56:11,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/1b4442c6b4f74fbeb67d25af2a52f77c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/1b4442c6b4f74fbeb67d25af2a52f77c 2023-11-10 04:56:11,711 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311107237c37e748f4268a1de2536c4187ed7_del 2023-11-10 04:56:11,711 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311107237c37e748f4268a1de2536c4187ed7_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202311107237c37e748f4268a1de2536c4187ed7_del 2023-11-10 04:56:12,524 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 01b0b5f002bb4789862353580085eceb 2023-11-10 04:56:12,525 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/.tmp/family2/01b0b5f002bb4789862353580085eceb as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/01b0b5f002bb4789862353580085eceb 2023-11-10 04:56:12,528 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 01b0b5f002bb4789862353580085eceb 2023-11-10 04:56:12,529 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in bc8052c221bbbcbdf08850f6e381c528/family2 of bc8052c221bbbcbdf08850f6e381c528 into 01b0b5f002bb4789862353580085eceb(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:56:12,529 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:56:12,529 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:56:12,529 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): f3cbc57e6f6fc77d5d5aa5038d824901/family1 is initiating major compaction (all files) 2023-11-10 04:56:12,529 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:56:12,529 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:56:12,529 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of f3cbc57e6f6fc77d5d5aa5038d824901/family1 in testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:12,530 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/a9ce51fca3f04401b843d92bf69498dc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/c8632ace49d442a98c8747ad89d96467, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/19a5037e0b7a48b0a90f6733b108d9db, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/63f4b67db25f475394530846b98731f5, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/05e8576a08d147ae8453929772103fe4] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp, totalSize=27.1 K 2023-11-10 04:56:12,530 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting a9ce51fca3f04401b843d92bf69498dc, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592158656 2023-11-10 04:56:12,530 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting c8632ace49d442a98c8747ad89d96467, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592160558 2023-11-10 04:56:12,530 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 19a5037e0b7a48b0a90f6733b108d9db, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592162451 2023-11-10 04:56:12,531 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 63f4b67db25f475394530846b98731f5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592164325 2023-11-10 04:56:12,531 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 05e8576a08d147ae8453929772103fe4, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:56:12,538 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): f3cbc57e6f6fc77d5d5aa5038d824901#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:56:12,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/eb5ea1487ea442f7821f02fa64fd915b, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/3f3cfd25000e435fb3bfa146ba400383, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/503b3bbcbf1643f2a9a8161e6e339609, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/f1e237ffc764469c9473d6784bfddf92, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/cf84e91255c0479a948fc94992476372] to archive 2023-11-10 04:56:12,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:12,577 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/eb5ea1487ea442f7821f02fa64fd915b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/eb5ea1487ea442f7821f02fa64fd915b 2023-11-10 04:56:12,578 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/3f3cfd25000e435fb3bfa146ba400383 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/3f3cfd25000e435fb3bfa146ba400383 2023-11-10 04:56:12,579 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/503b3bbcbf1643f2a9a8161e6e339609 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/503b3bbcbf1643f2a9a8161e6e339609 2023-11-10 04:56:12,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/f1e237ffc764469c9473d6784bfddf92 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/f1e237ffc764469c9473d6784bfddf92 2023-11-10 04:56:12,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/cf84e91255c0479a948fc94992476372 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family2/cf84e91255c0479a948fc94992476372 2023-11-10 04:56:12,780 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.15384615384615385, tune throughput to 57.69 MB/second 2023-11-10 04:56:12,975 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231110d3c42ce4b1854845ad97ce78d7ce9d89_del 2023-11-10 04:56:12,976 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110d3c42ce4b1854845ad97ce78d7ce9d89_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110d3c42ce4b1854845ad97ce78d7ce9d89_del 2023-11-10 04:56:13,644 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.15384615384615385, tune throughput to 57.69 MB/second 2023-11-10 04:56:13,830 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 204e454a344c4884ba4d478aea823447 2023-11-10 04:56:13,830 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family1/204e454a344c4884ba4d478aea823447 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/204e454a344c4884ba4d478aea823447 2023-11-10 04:56:13,852 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 204e454a344c4884ba4d478aea823447 2023-11-10 04:56:13,854 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f3cbc57e6f6fc77d5d5aa5038d824901/family1 of f3cbc57e6f6fc77d5d5aa5038d824901 into 204e454a344c4884ba4d478aea823447(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:56:13,854 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:56:13,854 DEBUG [Listener at localhost/35765] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-10 04:56:13,854 DEBUG [Listener at localhost/35765] regionserver.HStore(1912): f3cbc57e6f6fc77d5d5aa5038d824901/family2 is initiating major compaction (all files) 2023-11-10 04:56:13,854 INFO [Listener at localhost/35765] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-10 04:56:13,854 INFO [Listener at localhost/35765] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-10 04:56:13,854 INFO [Listener at localhost/35765] regionserver.HRegion(2259): Starting compaction of f3cbc57e6f6fc77d5d5aa5038d824901/family2 in testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:13,854 INFO [Listener at localhost/35765] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/ee4e76f5914b4ad681cc0d05abe33738, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/487c4d5b5c0a4345af1cf5cbd398d8ed, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/e04091a8d6154fad8235b9898c61a173, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/d89a000696c6452b9618a0e28824f2d7, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/fb921f44afb74539bcbf45d7ded1389d] into tmpdir=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp, totalSize=25.4 K 2023-11-10 04:56:13,855 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting ee4e76f5914b4ad681cc0d05abe33738, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1699592158656 2023-11-10 04:56:13,855 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting 487c4d5b5c0a4345af1cf5cbd398d8ed, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1699592160558 2023-11-10 04:56:13,855 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting e04091a8d6154fad8235b9898c61a173, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1699592162451 2023-11-10 04:56:13,856 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting d89a000696c6452b9618a0e28824f2d7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1699592164325 2023-11-10 04:56:13,856 DEBUG [Listener at localhost/35765] compactions.Compactor(207): Compacting fb921f44afb74539bcbf45d7ded1389d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-10 04:56:13,864 INFO [Listener at localhost/35765] throttle.PressureAwareThroughputController(145): f3cbc57e6f6fc77d5d5aa5038d824901#family2#compaction#202 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-10 04:56:13,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/a9ce51fca3f04401b843d92bf69498dc, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/c8632ace49d442a98c8747ad89d96467, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/19a5037e0b7a48b0a90f6733b108d9db, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/63f4b67db25f475394530846b98731f5, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/05e8576a08d147ae8453929772103fe4] to archive 2023-11-10 04:56:13,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:13,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/a9ce51fca3f04401b843d92bf69498dc to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/a9ce51fca3f04401b843d92bf69498dc 2023-11-10 04:56:13,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/c8632ace49d442a98c8747ad89d96467 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/c8632ace49d442a98c8747ad89d96467 2023-11-10 04:56:13,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/19a5037e0b7a48b0a90f6733b108d9db to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/19a5037e0b7a48b0a90f6733b108d9db 2023-11-10 04:56:13,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/63f4b67db25f475394530846b98731f5 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/63f4b67db25f475394530846b98731f5 2023-11-10 04:56:13,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/05e8576a08d147ae8453929772103fe4 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/05e8576a08d147ae8453929772103fe4 2023-11-10 04:56:14,272 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231110ab3f4e5cbc4b4e859e80a05d3e3adcaa_del 2023-11-10 04:56:14,272 INFO [Listener at localhost/35765] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231110ab3f4e5cbc4b4e859e80a05d3e3adcaa_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20231110ab3f4e5cbc4b4e859e80a05d3e3adcaa_del 2023-11-10 04:56:14,500 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.15384615384615385, tune throughput to 57.69 MB/second 2023-11-10 04:56:15,085 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ad246eea533a40b88e8fca51f4279476 2023-11-10 04:56:15,086 DEBUG [Listener at localhost/35765] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/.tmp/family2/ad246eea533a40b88e8fca51f4279476 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/ad246eea533a40b88e8fca51f4279476 2023-11-10 04:56:15,089 INFO [Listener at localhost/35765] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ad246eea533a40b88e8fca51f4279476 2023-11-10 04:56:15,090 INFO [Listener at localhost/35765] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f3cbc57e6f6fc77d5d5aa5038d824901/family2 of f3cbc57e6f6fc77d5d5aa5038d824901 into ad246eea533a40b88e8fca51f4279476(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-10 04:56:15,090 DEBUG [Listener at localhost/35765] regionserver.HRegion(2289): Compaction status journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:56:15,093 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:56:15,096 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:56:15,098 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:56:15,099 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:56:15,103 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46447] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10*~.\x01K\xBF@y)\xE7`\xF1!\xAB\xDC\xFA"\x10\xC6V\xB9$\xE13ZJ\xD8C\x15$w\xA04\x17*\x10O\xB1\xF8\xAF\x82\xD7\xA6D\xD7\x93\x07\x85\xB0\x08\x93A2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-11-10 04:56:15,124 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] procedure2.ProcedureExecutor(1028): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-10 04:56:15,125 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-10 04:56:15,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:56:15,126 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1699592775126 2023-11-10 04:56:15,126 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-10 04:56:15,181 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/ee4e76f5914b4ad681cc0d05abe33738, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/487c4d5b5c0a4345af1cf5cbd398d8ed, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/e04091a8d6154fad8235b9898c61a173, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/d89a000696c6452b9618a0e28824f2d7, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/fb921f44afb74539bcbf45d7ded1389d] to archive 2023-11-10 04:56:15,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:15,183 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/ee4e76f5914b4ad681cc0d05abe33738 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/ee4e76f5914b4ad681cc0d05abe33738 2023-11-10 04:56:15,184 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/487c4d5b5c0a4345af1cf5cbd398d8ed to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/487c4d5b5c0a4345af1cf5cbd398d8ed 2023-11-10 04:56:15,186 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/e04091a8d6154fad8235b9898c61a173 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/e04091a8d6154fad8235b9898c61a173 2023-11-10 04:56:15,187 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/d89a000696c6452b9618a0e28824f2d7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/d89a000696c6452b9618a0e28824f2d7 2023-11-10 04:56:15,188 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/fb921f44afb74539bcbf45d7ded1389d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family2/fb921f44afb74539bcbf45d7ded1389d 2023-11-10 04:56:15,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-11-10 04:56:15,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231110f64456ff97ce4b2f98a78bc56bb5fdb1_del 2023-11-10 04:56:15,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231110c4035e1acb844a3e86813278d66aa446_del 2023-11-10 04:56:15,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231110d3c42ce4b1854845ad97ce78d7ce9d89_del 2023-11-10 04:56:15,499 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-11-10 04:56:15,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] zookeeper.ReadOnlyZKClient(139): Connect 0x10f48eb4 to 127.0.0.1:51169 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-10 04:56:15,505 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e1d5328, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-10 04:56:15,505 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231110 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-10 04:56:16,490 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311103fa7665ac4e34643b088ae299b8f9fe9 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311103fa7665ac4e34643b088ae299b8f9fe9 2023-11-10 04:56:16,491 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='testMajorCompactionFromAdmin' column='family1' 2023-11-10 04:56:16,502 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-10 04:56:16,504 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48296, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-10 04:56:16,513 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-10 04:56:16,514 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56078, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-10 04:56:16,543 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231110/family1/e63f0e1faffd4e8789e8a123da1509d0 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-11-10 04:56:16,543 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231110/family1/e63f0e1faffd4e8789e8a123da1509d0}] 2023-11-10 04:56:16,548 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231110/family1/e63f0e1faffd4e8789e8a123da1509d0 for inclusion in f3cbc57e6f6fc77d5d5aa5038d824901/family1 2023-11-10 04:56:16,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-11-10 04:56:16,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-10 04:56:16,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:56:16,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231110/family1/e63f0e1faffd4e8789e8a123da1509d0 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__94mvic81gi3472jqc3lvs413e88dmcbt7b06i25ipgtibfaf4l23cim57q77spie/family1/e63f0e1faffd4e8789e8a123da1509d0 2023-11-10 04:56:16,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__94mvic81gi3472jqc3lvs413e88dmcbt7b06i25ipgtibfaf4l23cim57q77spie/family1/e63f0e1faffd4e8789e8a123da1509d0 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/07f12e79d67a42879d3030f6cb2b16b2_SeqId_28_ 2023-11-10 04:56:16,575 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__94mvic81gi3472jqc3lvs413e88dmcbt7b06i25ipgtibfaf4l23cim57q77spie/family1/e63f0e1faffd4e8789e8a123da1509d0 into f3cbc57e6f6fc77d5d5aa5038d824901/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/07f12e79d67a42879d3030f6cb2b16b2_SeqId_28_ - updating store file list. 2023-11-10 04:56:16,596 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/07f12e79d67a42879d3030f6cb2b16b2_SeqId_28_ into f3cbc57e6f6fc77d5d5aa5038d824901/family1 2023-11-10 04:56:16,596 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__94mvic81gi3472jqc3lvs413e88dmcbt7b06i25ipgtibfaf4l23cim57q77spie/family1/e63f0e1faffd4e8789e8a123da1509d0 into f3cbc57e6f6fc77d5d5aa5038d824901/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/family1/07f12e79d67a42879d3030f6cb2b16b2_SeqId_28_) 2023-11-10 04:56:16,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__94mvic81gi3472jqc3lvs413e88dmcbt7b06i25ipgtibfaf4l23cim57q77spie/family1/e63f0e1faffd4e8789e8a123da1509d0 2023-11-10 04:56:16,598 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231110 table='testMajorCompactionFromAdmin' column='family1' 2023-11-10 04:56:16,599 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:16,600 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110813447ec2cf74d5ba6529ab59cf1d8cf to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110813447ec2cf74d5ba6529ab59cf1d8cf 2023-11-10 04:56:16,601 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023111084f139d451a64ad0a821d43d5623cbf3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023111084f139d451a64ad0a821d43d5623cbf3 2023-11-10 04:56:16,602 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110a02a0a7884894f18bfa41a9299f9f124 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110a02a0a7884894f18bfa41a9299f9f124 2023-11-10 04:56:16,603 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110def95f5928c745e98b81bf0448fc8f8d to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110def95f5928c745e98b81bf0448fc8f8d 2023-11-10 04:56:16,603 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-10 04:56:16,603 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231110' 2023-11-10 04:56:16,603 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231110 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-10 04:56:16,603 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231110 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-10 04:56:17,545 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311101a446fbebe794ff08fee2146cfbe41f3 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311101a446fbebe794ff08fee2146cfbe41f3 2023-11-10 04:56:17,546 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231110 table='testMajorCompactionFromAdmin' column='family1' 2023-11-10 04:56:17,585 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231110/family1/480f4a4de05645259e087f0a92368d22 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-11-10 04:56:17,586 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231110/family1/480f4a4de05645259e087f0a92368d22}] 2023-11-10 04:56:17,588 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231110/family1/480f4a4de05645259e087f0a92368d22 for inclusion in 8b858392c9cfe904206ed137d6282aa0/family1 2023-11-10 04:56:17,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-11-10 04:56:17,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-10 04:56:17,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:56:17,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231110/family1/480f4a4de05645259e087f0a92368d22 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__2gghpttrrt5kbl3vgl8plbrq1d7e7ufvm7uu8mj0mdp7nb8tvpesr84t6kf199vq/family1/480f4a4de05645259e087f0a92368d22 2023-11-10 04:56:17,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__2gghpttrrt5kbl3vgl8plbrq1d7e7ufvm7uu8mj0mdp7nb8tvpesr84t6kf199vq/family1/480f4a4de05645259e087f0a92368d22 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/c2ad4fd2847d4ec1ad1d608bb539c943_SeqId_28_ 2023-11-10 04:56:17,611 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__2gghpttrrt5kbl3vgl8plbrq1d7e7ufvm7uu8mj0mdp7nb8tvpesr84t6kf199vq/family1/480f4a4de05645259e087f0a92368d22 into 8b858392c9cfe904206ed137d6282aa0/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/c2ad4fd2847d4ec1ad1d608bb539c943_SeqId_28_ - updating store file list. 2023-11-10 04:56:17,631 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/c2ad4fd2847d4ec1ad1d608bb539c943_SeqId_28_ into 8b858392c9cfe904206ed137d6282aa0/family1 2023-11-10 04:56:17,631 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__2gghpttrrt5kbl3vgl8plbrq1d7e7ufvm7uu8mj0mdp7nb8tvpesr84t6kf199vq/family1/480f4a4de05645259e087f0a92368d22 into 8b858392c9cfe904206ed137d6282aa0/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/family1/c2ad4fd2847d4ec1ad1d608bb539c943_SeqId_28_) 2023-11-10 04:56:17,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__2gghpttrrt5kbl3vgl8plbrq1d7e7ufvm7uu8mj0mdp7nb8tvpesr84t6kf199vq/family1/480f4a4de05645259e087f0a92368d22 2023-11-10 04:56:17,634 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231110 table='testMajorCompactionFromAdmin' column='family1' 2023-11-10 04:56:17,634 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:17,636 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110a4a3a96209514de1bc3de48d349d1943 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110a4a3a96209514de1bc3de48d349d1943 2023-11-10 04:56:17,637 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110d82ebaec8b324c65aa2a23881c77b43c to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110d82ebaec8b324c65aa2a23881c77b43c 2023-11-10 04:56:17,638 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110e766efefe0044f27b44594d81bd3fb24 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110e766efefe0044f27b44594d81bd3fb24 2023-11-10 04:56:17,638 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110edd66af8ef84455383303ac73a797f2b to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110edd66af8ef84455383303ac73a797f2b 2023-11-10 04:56:17,639 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-10 04:56:17,639 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231110' 2023-11-10 04:56:17,639 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231110 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-10 04:56:17,639 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231110 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-10 04:56:18,604 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023111080204b06ab9248bfafb6dae5cb85c405 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023111080204b06ab9248bfafb6dae5cb85c405 2023-11-10 04:56:18,605 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='testMajorCompactionFromAdmin' column='family1' 2023-11-10 04:56:18,644 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231110/family1/df4f4d92055d4719824a370cee5fa5e7 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-11-10 04:56:18,644 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528., hostname=jenkins-hbase4.apache.org,36419,1699592002485, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231110/family1/df4f4d92055d4719824a370cee5fa5e7}] 2023-11-10 04:56:18,646 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(763): Validating hfile at hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231110/family1/df4f4d92055d4719824a370cee5fa5e7 for inclusion in bc8052c221bbbcbdf08850f6e381c528/family1 2023-11-10 04:56:18,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-11-10 04:56:18,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(777): Region bounds: first= last=1 2023-11-10 04:56:18,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegion(2446): Flush status journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:56:18,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231110/family1/df4f4d92055d4719824a370cee5fa5e7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__h295mgf0glh00v0fkv9s0r7e8ji66q8peitflib27p7q3pool2h2oa6uvpk113g3/family1/df4f4d92055d4719824a370cee5fa5e7 2023-11-10 04:56:18,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__h295mgf0glh00v0fkv9s0r7e8ji66q8peitflib27p7q3pool2h2oa6uvpk113g3/family1/df4f4d92055d4719824a370cee5fa5e7 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/cdb656894bb24125b335fd4092d976af_SeqId_28_ 2023-11-10 04:56:18,669 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(849): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__h295mgf0glh00v0fkv9s0r7e8ji66q8peitflib27p7q3pool2h2oa6uvpk113g3/family1/df4f4d92055d4719824a370cee5fa5e7 into bc8052c221bbbcbdf08850f6e381c528/family1 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/cdb656894bb24125b335fd4092d976af_SeqId_28_ - updating store file list. 2023-11-10 04:56:18,702 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(882): Loaded HFile hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/cdb656894bb24125b335fd4092d976af_SeqId_28_ into bc8052c221bbbcbdf08850f6e381c528/family1 2023-11-10 04:56:18,702 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.HStore(855): Successfully loaded hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__h295mgf0glh00v0fkv9s0r7e8ji66q8peitflib27p7q3pool2h2oa6uvpk113g3/family1/df4f4d92055d4719824a370cee5fa5e7 into bc8052c221bbbcbdf08850f6e381c528/family1 (new location: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/family1/cdb656894bb24125b335fd4092d976af_SeqId_28_) 2023-11-10 04:56:18,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36419] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/staging/jenkins__testMajorCompactionFromAdmin__h295mgf0glh00v0fkv9s0r7e8ji66q8peitflib27p7q3pool2h2oa6uvpk113g3/family1/df4f4d92055d4719824a370cee5fa5e7 2023-11-10 04:56:18,705 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231110 table='testMajorCompactionFromAdmin' column='family1' 2023-11-10 04:56:18,705 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:18,707 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110206e6f8b071e47b384bf70c6b05ce3b7 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110206e6f8b071e47b384bf70c6b05ce3b7 2023-11-10 04:56:18,708 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110289a29f93eaa416aaac3ad05f66e9770 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110289a29f93eaa416aaac3ad05f66e9770 2023-11-10 04:56:18,709 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110927ec0cd8e2945709d7b833aaebeff38 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110927ec0cd8e2945709d7b833aaebeff38 2023-11-10 04:56:18,710 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110f4c4436d045947c59bc20bc6114bc166 to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110f4c4436d045947c59bc20bc6114bc166 2023-11-10 04:56:18,710 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-10 04:56:18,710 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231110' 2023-11-10 04:56:18,710 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231110 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-10 04:56:18,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-11-10 04:56:18,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-10 04:56:18,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110f64456ff97ce4b2f98a78bc56bb5fdb1_del] 2023-11-10 04:56:18,711 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:18,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110f64456ff97ce4b2f98a78bc56bb5fdb1_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231110f64456ff97ce4b2f98a78bc56bb5fdb1_del 2023-11-10 04:56:18,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-10 04:56:18,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110c4035e1acb844a3e86813278d66aa446_del] 2023-11-10 04:56:18,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:18,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110c4035e1acb844a3e86813278d66aa446_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231110c4035e1acb844a3e86813278d66aa446_del 2023-11-10 04:56:18,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-10 04:56:18,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110d3c42ce4b1854845ad97ce78d7ce9d89_del] 2023-11-10 04:56:18,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] backup.HFileArchiver(360): Archiving compacted files. 2023-11-10 04:56:18,715 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110d3c42ce4b1854845ad97ce78d7ce9d89_del to hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231110d3c42ce4b1854845ad97ce78d7ce9d89_del 2023-11-10 04:56:18,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-10 04:56:18,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1699592007144] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-10 04:56:18,716 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-10 04:56:18,717 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.5930 sec 2023-11-10 04:56:18,729 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:56:18,731 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:56:18,733 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-10 04:56:18,734 DEBUG [Listener at localhost/35765] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-10 04:56:18,737 INFO [Listener at localhost/35765] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-10 04:56:18,771 INFO [Listener at localhost/35765] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=249 (was 261), OpenFileDescriptor=652 (was 680), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=126 (was 127), ProcessCount=167 (was 167), AvailableMemoryMB=11098 (was 11116) 2023-11-10 04:56:18,772 DEBUG [Listener at localhost/35765] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x05ae2dad to 127.0.0.1:51169 2023-11-10 04:56:18,772 DEBUG [Listener at localhost/35765] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:56:18,772 INFO [Listener at localhost/35765] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-11-10 04:56:18,772 INFO [Listener at localhost/35765] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-10 04:56:18,772 DEBUG [Listener at localhost/35765] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2324da13 to 127.0.0.1:51169 2023-11-10 04:56:18,773 DEBUG [Listener at localhost/35765] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:56:18,773 DEBUG [Listener at localhost/35765] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-11-10 04:56:18,773 DEBUG [Listener at localhost/35765] util.JVMClusterUtil(257): Found active master hash=922630575, stopped=false 2023-11-10 04:56:18,773 INFO [Listener at localhost/35765] master.ServerManager(888): Cluster shutdown requested of master=jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:56:18,775 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-10 04:56:18,775 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-10 04:56:18,775 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:56:18,775 INFO [Listener at localhost/35765] procedure2.ProcedureExecutor(628): Stopping 2023-11-10 04:56:18,776 DEBUG [Listener at localhost/35765] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x295edb9d to 127.0.0.1:51169 2023-11-10 04:56:18,776 DEBUG [Listener at localhost/35765] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:56:18,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-10 04:56:18,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-10 04:56:18,776 INFO [Listener at localhost/35765] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36419,1699592002485' ***** 2023-11-10 04:56:18,776 INFO [Listener at localhost/35765] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-11-10 04:56:18,776 INFO [RS:0;jenkins-hbase4:36419] regionserver.HeapMemoryManager(220): Stopping 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-11-10 04:56:18,777 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 3d66950151f7b8ec803595548ad6d16c 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for d16400a47b40f4a34c475d374e4521a1 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for a5bb60bfaec2aed551dc765427373c48 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 8b858392c9cfe904206ed137d6282aa0 2023-11-10 04:56:18,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3d66950151f7b8ec803595548ad6d16c, disabling compactions & flushes 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for bc8052c221bbbcbdf08850f6e381c528 2023-11-10 04:56:18,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 428db5f998f281799ebc91a16e0c51d8 2023-11-10 04:56:18,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for d7008324b61f95fffabeee9f0b675863 2023-11-10 04:56:18,777 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 422f46c0e9462ab44880fe4aeb8fb7a2 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 92e368b193b9f67ef8fc1cb88c465c29 2023-11-10 04:56:18,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. after waiting 0 ms 2023-11-10 04:56:18,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for cbac8c6d54c66bf2d99bd01264a1e1ca 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for d0cb33d1bbec7f17d50fcde1f31e585e 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 8ceeb2db4dd461c2eab386dd8a01d75d 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for d4d1a475f6a44a306a5cc4d860a9f915 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 247092fa363b36c6fef420107a683723 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for cc06acced8f53930ee1704b8e090b9b9 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for ee1662c8ccd6e1d2468c8603ba66a3a6 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 49d033c6fa4701f6eda4cca344a97853 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 2b8f43a41c5d5687d6c3e84194ded34f 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 39a447f61d308d5a04d02b6b0b63c0b3 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:56:18,778 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:18,778 DEBUG [RS:0;jenkins-hbase4:36419] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x04ff47a3 to 127.0.0.1:51169 2023-11-10 04:56:18,778 DEBUG [RS:0;jenkins-hbase4:36419] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:56:18,779 INFO [RS:0;jenkins-hbase4:36419] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-11-10 04:56:18,779 INFO [RS:0;jenkins-hbase4:36419] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-11-10 04:56:18,779 INFO [RS:0;jenkins-hbase4:36419] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-11-10 04:56:18,779 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-11-10 04:56:18,779 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1474): Waiting on 21 regions to close 2023-11-10 04:56:18,779 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-10 04:56:18,779 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-10 04:56:18,779 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-10 04:56:18,779 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-10 04:56:18,779 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-10 04:56:18,779 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-11-10 04:56:18,780 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1504): Waiting on 1588230740, 247092fa363b36c6fef420107a683723, 2b8f43a41c5d5687d6c3e84194ded34f, 39a447f61d308d5a04d02b6b0b63c0b3, 3d66950151f7b8ec803595548ad6d16c, 422f46c0e9462ab44880fe4aeb8fb7a2, 428db5f998f281799ebc91a16e0c51d8, 49d033c6fa4701f6eda4cca344a97853, 8b858392c9cfe904206ed137d6282aa0, 8ceeb2db4dd461c2eab386dd8a01d75d, 92e368b193b9f67ef8fc1cb88c465c29, a5bb60bfaec2aed551dc765427373c48, bc8052c221bbbcbdf08850f6e381c528, cbac8c6d54c66bf2d99bd01264a1e1ca, cc06acced8f53930ee1704b8e090b9b9, d0cb33d1bbec7f17d50fcde1f31e585e, d16400a47b40f4a34c475d374e4521a1, d4d1a475f6a44a306a5cc4d860a9f915, d7008324b61f95fffabeee9f0b675863, ee1662c8ccd6e1d2468c8603ba66a3a6, f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:56:18,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithMonthlyPolicy/3d66950151f7b8ec803595548ad6d16c/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-11-10 04:56:18,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:56:18,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3d66950151f7b8ec803595548ad6d16c: 2023-11-10 04:56:18,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1699592019321.3d66950151f7b8ec803595548ad6d16c. 2023-11-10 04:56:18,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d16400a47b40f4a34c475d374e4521a1, disabling compactions & flushes 2023-11-10 04:56:18,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:56:18,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:56:18,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. after waiting 0 ms 2023-11-10 04:56:18,783 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:56:18,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/d16400a47b40f4a34c475d374e4521a1/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-10 04:56:18,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:56:18,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d16400a47b40f4a34c475d374e4521a1: 2023-11-10 04:56:18,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1699592125540.d16400a47b40f4a34c475d374e4521a1. 2023-11-10 04:56:18,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a5bb60bfaec2aed551dc765427373c48, disabling compactions & flushes 2023-11-10 04:56:18,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:56:18,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:56:18,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. after waiting 0 ms 2023-11-10 04:56:18,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:56:18,790 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/.tmp/info/ee6866e0d128496f828901a5710cb034 2023-11-10 04:56:18,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/a5bb60bfaec2aed551dc765427373c48/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-11-10 04:56:18,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:56:18,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a5bb60bfaec2aed551dc765427373c48: 2023-11-10 04:56:18,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1699592045556.a5bb60bfaec2aed551dc765427373c48. 2023-11-10 04:56:18,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8b858392c9cfe904206ed137d6282aa0, disabling compactions & flushes 2023-11-10 04:56:18,793 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:18,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:18,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. after waiting 0 ms 2023-11-10 04:56:18,793 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:18,793 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee6866e0d128496f828901a5710cb034 2023-11-10 04:56:18,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/8b858392c9cfe904206ed137d6282aa0/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-10 04:56:18,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:18,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8b858392c9cfe904206ed137d6282aa0: 2023-11-10 04:56:18,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1699592156497.8b858392c9cfe904206ed137d6282aa0. 2023-11-10 04:56:18,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bc8052c221bbbcbdf08850f6e381c528, disabling compactions & flushes 2023-11-10 04:56:18,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:18,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:18,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. after waiting 0 ms 2023-11-10 04:56:18,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:18,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/bc8052c221bbbcbdf08850f6e381c528/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-10 04:56:18,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:18,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bc8052c221bbbcbdf08850f6e381c528: 2023-11-10 04:56:18,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1699592156497.bc8052c221bbbcbdf08850f6e381c528. 2023-11-10 04:56:18,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 428db5f998f281799ebc91a16e0c51d8, disabling compactions & flushes 2023-11-10 04:56:18,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:56:18,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:56:18,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. after waiting 0 ms 2023-11-10 04:56:18,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:56:18,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/428db5f998f281799ebc91a16e0c51d8/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-10 04:56:18,805 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:56:18,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 428db5f998f281799ebc91a16e0c51d8: 2023-11-10 04:56:18,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1699592101202.428db5f998f281799ebc91a16e0c51d8. 2023-11-10 04:56:18,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d7008324b61f95fffabeee9f0b675863, disabling compactions & flushes 2023-11-10 04:56:18,805 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:56:18,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:56:18,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. after waiting 0 ms 2023-11-10 04:56:18,805 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:56:18,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithMonthlyPolicy/d7008324b61f95fffabeee9f0b675863/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-11-10 04:56:18,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:56:18,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d7008324b61f95fffabeee9f0b675863: 2023-11-10 04:56:18,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1699592143106.d7008324b61f95fffabeee9f0b675863. 2023-11-10 04:56:18,808 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 422f46c0e9462ab44880fe4aeb8fb7a2, disabling compactions & flushes 2023-11-10 04:56:18,808 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:56:18,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:56:18,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. after waiting 0 ms 2023-11-10 04:56:18,809 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:56:18,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMinorCompactionWithWeeklyPolicy/422f46c0e9462ab44880fe4aeb8fb7a2/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-10 04:56:18,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:56:18,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 422f46c0e9462ab44880fe4aeb8fb7a2: 2023-11-10 04:56:18,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1699592032249.422f46c0e9462ab44880fe4aeb8fb7a2. 2023-11-10 04:56:18,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 92e368b193b9f67ef8fc1cb88c465c29, disabling compactions & flushes 2023-11-10 04:56:18,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:56:18,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:56:18,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. after waiting 0 ms 2023-11-10 04:56:18,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:56:18,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyPolicy/92e368b193b9f67ef8fc1cb88c465c29/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-11-10 04:56:18,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:56:18,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 92e368b193b9f67ef8fc1cb88c465c29: 2023-11-10 04:56:18,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1699592049576.92e368b193b9f67ef8fc1cb88c465c29. 2023-11-10 04:56:18,815 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cbac8c6d54c66bf2d99bd01264a1e1ca, disabling compactions & flushes 2023-11-10 04:56:18,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:56:18,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:56:18,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. after waiting 0 ms 2023-11-10 04:56:18,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:56:18,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/cbac8c6d54c66bf2d99bd01264a1e1ca/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-11-10 04:56:18,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:56:18,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cbac8c6d54c66bf2d99bd01264a1e1ca: 2023-11-10 04:56:18,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1699592045556.cbac8c6d54c66bf2d99bd01264a1e1ca. 2023-11-10 04:56:18,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d0cb33d1bbec7f17d50fcde1f31e585e, disabling compactions & flushes 2023-11-10 04:56:18,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:56:18,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:56:18,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. after waiting 0 ms 2023-11-10 04:56:18,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:56:18,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/d0cb33d1bbec7f17d50fcde1f31e585e/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-11-10 04:56:18,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:56:18,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d0cb33d1bbec7f17d50fcde1f31e585e: 2023-11-10 04:56:18,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1699592007363.d0cb33d1bbec7f17d50fcde1f31e585e. 2023-11-10 04:56:18,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8ceeb2db4dd461c2eab386dd8a01d75d, disabling compactions & flushes 2023-11-10 04:56:18,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:56:18,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:56:18,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. after waiting 0 ms 2023-11-10 04:56:18,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:56:18,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/8ceeb2db4dd461c2eab386dd8a01d75d/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-10 04:56:18,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:56:18,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8ceeb2db4dd461c2eab386dd8a01d75d: 2023-11-10 04:56:18,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1699592125540.8ceeb2db4dd461c2eab386dd8a01d75d. 2023-11-10 04:56:18,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d4d1a475f6a44a306a5cc4d860a9f915, disabling compactions & flushes 2023-11-10 04:56:18,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:56:18,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:56:18,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. after waiting 0 ms 2023-11-10 04:56:18,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:56:18,828 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testScannerOnBulkLoadRefHFiles/d4d1a475f6a44a306a5cc4d860a9f915/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-11-10 04:56:18,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:56:18,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d4d1a475f6a44a306a5cc4d860a9f915: 2023-11-10 04:56:18,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1699592045556.d4d1a475f6a44a306a5cc4d860a9f915. 2023-11-10 04:56:18,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 247092fa363b36c6fef420107a683723, disabling compactions & flushes 2023-11-10 04:56:18,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:56:18,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:56:18,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. after waiting 0 ms 2023-11-10 04:56:18,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:56:18,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/247092fa363b36c6fef420107a683723/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-10 04:56:18,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:56:18,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 247092fa363b36c6fef420107a683723: 2023-11-10 04:56:18,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1699592101202.247092fa363b36c6fef420107a683723. 2023-11-10 04:56:18,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cc06acced8f53930ee1704b8e090b9b9, disabling compactions & flushes 2023-11-10 04:56:18,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:56:18,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:56:18,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. after waiting 0 ms 2023-11-10 04:56:18,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:56:18,837 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/ns/testMinorCompaction/cc06acced8f53930ee1704b8e090b9b9/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-10 04:56:18,838 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:56:18,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cc06acced8f53930ee1704b8e090b9b9: 2023-11-10 04:56:18,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1699592125540.cc06acced8f53930ee1704b8e090b9b9. 2023-11-10 04:56:18,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ee1662c8ccd6e1d2468c8603ba66a3a6, disabling compactions & flushes 2023-11-10 04:56:18,838 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:56:18,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:56:18,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. after waiting 0 ms 2023-11-10 04:56:18,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:56:18,838 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-11-10 04:56:18,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testCompactionWithHFileLink/ee1662c8ccd6e1d2468c8603ba66a3a6/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-10 04:56:18,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:56:18,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ee1662c8ccd6e1d2468c8603ba66a3a6: 2023-11-10 04:56:18,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1699592101202.ee1662c8ccd6e1d2468c8603ba66a3a6. 2023-11-10 04:56:18,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 49d033c6fa4701f6eda4cca344a97853, disabling compactions & flushes 2023-11-10 04:56:18,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:56:18,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:56:18,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. after waiting 0 ms 2023-11-10 04:56:18,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:56:18,841 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 49d033c6fa4701f6eda4cca344a97853 1/1 column families, dataSize=109 B heapSize=592 B 2023-11-10 04:56:18,879 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-11-10 04:56:18,980 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1504): Waiting on 1588230740, 2b8f43a41c5d5687d6c3e84194ded34f, 39a447f61d308d5a04d02b6b0b63c0b3, 49d033c6fa4701f6eda4cca344a97853, f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:56:19,181 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1504): Waiting on 1588230740, 2b8f43a41c5d5687d6c3e84194ded34f, 39a447f61d308d5a04d02b6b0b63c0b3, 49d033c6fa4701f6eda4cca344a97853, f3cbc57e6f6fc77d5d5aa5038d824901 2023-11-10 04:56:19,200 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/.tmp/rep_barrier/237838d040264208b6b050e1fc470d3b 2023-11-10 04:56:19,205 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 237838d040264208b6b050e1fc470d3b 2023-11-10 04:56:19,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853/.tmp/info/95bcdb75cbb6465c9aab8a2039adecf5 2023-11-10 04:56:19,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853/.tmp/info/95bcdb75cbb6465c9aab8a2039adecf5 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853/info/95bcdb75cbb6465c9aab8a2039adecf5 2023-11-10 04:56:19,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853/info/95bcdb75cbb6465c9aab8a2039adecf5, entries=3, sequenceid=7, filesize=4.8 K 2023-11-10 04:56:19,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 49d033c6fa4701f6eda4cca344a97853 in 415ms, sequenceid=7, compaction requested=false 2023-11-10 04:56:19,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/namespace/49d033c6fa4701f6eda4cca344a97853/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-11-10 04:56:19,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:56:19,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 49d033c6fa4701f6eda4cca344a97853: 2023-11-10 04:56:19,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1699592006027.49d033c6fa4701f6eda4cca344a97853. 2023-11-10 04:56:19,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2b8f43a41c5d5687d6c3e84194ded34f, disabling compactions & flushes 2023-11-10 04:56:19,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:56:19,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:56:19,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. after waiting 0 ms 2023-11-10 04:56:19,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:56:19,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/2b8f43a41c5d5687d6c3e84194ded34f/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-11-10 04:56:19,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:56:19,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2b8f43a41c5d5687d6c3e84194ded34f: 2023-11-10 04:56:19,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1699592078674.2b8f43a41c5d5687d6c3e84194ded34f. 2023-11-10 04:56:19,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 39a447f61d308d5a04d02b6b0b63c0b3, disabling compactions & flushes 2023-11-10 04:56:19,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:56:19,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:56:19,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. after waiting 0 ms 2023-11-10 04:56:19,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:56:19,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/39a447f61d308d5a04d02b6b0b63c0b3/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-11-10 04:56:19,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:56:19,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 39a447f61d308d5a04d02b6b0b63c0b3: 2023-11-10 04:56:19,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1699592061651.39a447f61d308d5a04d02b6b0b63c0b3. 2023-11-10 04:56:19,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f3cbc57e6f6fc77d5d5aa5038d824901, disabling compactions & flushes 2023-11-10 04:56:19,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:19,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:19,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. after waiting 0 ms 2023-11-10 04:56:19,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:19,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/default/testMajorCompactionFromAdmin/f3cbc57e6f6fc77d5d5aa5038d824901/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-10 04:56:19,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:19,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f3cbc57e6f6fc77d5d5aa5038d824901: 2023-11-10 04:56:19,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1699592156497.f3cbc57e6f6fc77d5d5aa5038d824901. 2023-11-10 04:56:19,381 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-11-10 04:56:19,581 DEBUG [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-11-10 04:56:19,611 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/.tmp/table/32a1c98135274b928d5fbdd0f0a25d48 2023-11-10 04:56:19,614 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 32a1c98135274b928d5fbdd0f0a25d48 2023-11-10 04:56:19,614 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/.tmp/info/ee6866e0d128496f828901a5710cb034 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/info/ee6866e0d128496f828901a5710cb034 2023-11-10 04:56:19,617 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee6866e0d128496f828901a5710cb034 2023-11-10 04:56:19,618 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/info/ee6866e0d128496f828901a5710cb034, entries=226, sequenceid=130, filesize=37.9 K 2023-11-10 04:56:19,618 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/.tmp/rep_barrier/237838d040264208b6b050e1fc470d3b as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/rep_barrier/237838d040264208b6b050e1fc470d3b 2023-11-10 04:56:19,621 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 237838d040264208b6b050e1fc470d3b 2023-11-10 04:56:19,621 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/rep_barrier/237838d040264208b6b050e1fc470d3b, entries=1, sequenceid=130, filesize=5.0 K 2023-11-10 04:56:19,622 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/.tmp/table/32a1c98135274b928d5fbdd0f0a25d48 as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/table/32a1c98135274b928d5fbdd0f0a25d48 2023-11-10 04:56:19,625 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 32a1c98135274b928d5fbdd0f0a25d48 2023-11-10 04:56:19,625 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/table/32a1c98135274b928d5fbdd0f0a25d48, entries=27, sequenceid=130, filesize=6.6 K 2023-11-10 04:56:19,627 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 848ms, sequenceid=130, compaction requested=false 2023-11-10 04:56:19,630 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-11-10 04:56:19,631 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-10 04:56:19,631 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-10 04:56:19,631 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-10 04:56:19,631 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-11-10 04:56:19,698 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-11-10 04:56:19,698 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-11-10 04:56:19,781 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36419,1699592002485; all regions closed. 2023-11-10 04:56:19,789 DEBUG [RS:0;jenkins-hbase4:36419] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/oldWALs 2023-11-10 04:56:19,789 INFO [RS:0;jenkins-hbase4:36419] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36419%2C1699592002485.meta:.meta(num 1699592005764) 2023-11-10 04:56:19,793 DEBUG [RS:0;jenkins-hbase4:36419] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/oldWALs 2023-11-10 04:56:19,793 INFO [RS:0;jenkins-hbase4:36419] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36419%2C1699592002485:(num 1699592004853) 2023-11-10 04:56:19,793 DEBUG [RS:0;jenkins-hbase4:36419] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:56:19,793 INFO [RS:0;jenkins-hbase4:36419] regionserver.LeaseManager(133): Closed leases 2023-11-10 04:56:19,793 INFO [RS:0;jenkins-hbase4:36419] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-11-10 04:56:19,794 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-10 04:56:19,795 INFO [RS:0;jenkins-hbase4:36419] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36419 2023-11-10 04:56:19,804 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36419,1699592002485 2023-11-10 04:56:19,804 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-10 04:56:19,804 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-10 04:56:19,806 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36419,1699592002485] 2023-11-10 04:56:19,806 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36419,1699592002485; numProcessing=1 2023-11-10 04:56:19,807 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36419,1699592002485 already deleted, retry=false 2023-11-10 04:56:19,807 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase4.apache.org,36419,1699592002485 expired; onlineServers=0 2023-11-10 04:56:19,807 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46447,1699592001315' ***** 2023-11-10 04:56:19,807 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-11-10 04:56:19,808 DEBUG [M:0;jenkins-hbase4:46447] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77c33394, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-10 04:56:19,808 INFO [M:0;jenkins-hbase4:46447] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46447,1699592001315 2023-11-10 04:56:19,808 INFO [M:0;jenkins-hbase4:46447] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46447,1699592001315; all regions closed. 2023-11-10 04:56:19,808 DEBUG [M:0;jenkins-hbase4:46447] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-10 04:56:19,808 DEBUG [M:0;jenkins-hbase4:46447] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-11-10 04:56:19,808 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-11-10 04:56:19,808 DEBUG [M:0;jenkins-hbase4:46447] cleaner.HFileCleaner(317): Stopping file delete threads 2023-11-10 04:56:19,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1699592004496] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1699592004496,5,FailOnTimeoutGroup] 2023-11-10 04:56:19,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1699592004495] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1699592004495,5,FailOnTimeoutGroup] 2023-11-10 04:56:19,809 INFO [M:0;jenkins-hbase4:46447] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-11-10 04:56:19,809 INFO [M:0;jenkins-hbase4:46447] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-11-10 04:56:19,809 INFO [M:0;jenkins-hbase4:46447] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-11-10 04:56:19,809 DEBUG [M:0;jenkins-hbase4:46447] master.HMaster(1512): Stopping service threads 2023-11-10 04:56:19,809 INFO [M:0;jenkins-hbase4:46447] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-11-10 04:56:19,809 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-11-10 04:56:19,809 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-10 04:56:19,810 ERROR [M:0;jenkins-hbase4:46447] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-11-10 04:56:19,810 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-10 04:56:19,810 INFO [M:0;jenkins-hbase4:46447] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-10 04:56:19,810 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-11-10 04:56:19,811 DEBUG [M:0;jenkins-hbase4:46447] zookeeper.ZKUtil(398): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-11-10 04:56:19,811 WARN [M:0;jenkins-hbase4:46447] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-11-10 04:56:19,811 INFO [M:0;jenkins-hbase4:46447] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-10 04:56:19,811 INFO [M:0;jenkins-hbase4:46447] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-11-10 04:56:19,811 DEBUG [M:0;jenkins-hbase4:46447] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-10 04:56:19,811 INFO [M:0;jenkins-hbase4:46447] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-10 04:56:19,811 DEBUG [M:0;jenkins-hbase4:46447] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-10 04:56:19,811 DEBUG [M:0;jenkins-hbase4:46447] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-10 04:56:19,811 DEBUG [M:0;jenkins-hbase4:46447] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-10 04:56:19,811 INFO [M:0;jenkins-hbase4:46447] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.78 KB heapSize=598.59 KB 2023-11-10 04:56:19,906 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-10 04:56:19,906 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): regionserver:36419-0x101ce530b520001, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-10 04:56:19,906 INFO [RS:0;jenkins-hbase4:36419] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36419,1699592002485; zookeeper connection closed. 2023-11-10 04:56:19,907 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2816944a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2816944a 2023-11-10 04:56:19,907 INFO [Listener at localhost/35765] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-11-10 04:56:20,218 INFO [M:0;jenkins-hbase4:46447] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.78 KB at sequenceid=1006 (bloomFilter=true), to=hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/498764ded9864d72a9e7a984c2089b7a 2023-11-10 04:56:20,221 INFO [M:0;jenkins-hbase4:46447] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 498764ded9864d72a9e7a984c2089b7a 2023-11-10 04:56:20,222 DEBUG [M:0;jenkins-hbase4:46447] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/498764ded9864d72a9e7a984c2089b7a as hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/498764ded9864d72a9e7a984c2089b7a 2023-11-10 04:56:20,225 INFO [M:0;jenkins-hbase4:46447] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 498764ded9864d72a9e7a984c2089b7a 2023-11-10 04:56:20,225 INFO [M:0;jenkins-hbase4:46447] regionserver.HStore(1080): Added hdfs://localhost:43735/user/jenkins/test-data/a7348012-65f4-4c51-0007-b5833f302af8/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/498764ded9864d72a9e7a984c2089b7a, entries=123, sequenceid=1006, filesize=33.3 K 2023-11-10 04:56:20,226 INFO [M:0;jenkins-hbase4:46447] regionserver.HRegion(2948): Finished flush of dataSize ~512.78 KB/525089, heapSize ~598.57 KB/612936, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1006, compaction requested=false 2023-11-10 04:56:20,226 INFO [M:0;jenkins-hbase4:46447] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-10 04:56:20,226 DEBUG [M:0;jenkins-hbase4:46447] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-10 04:56:20,229 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-10 04:56:20,229 INFO [M:0;jenkins-hbase4:46447] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-11-10 04:56:20,230 INFO [M:0;jenkins-hbase4:46447] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46447 2023-11-10 04:56:20,233 DEBUG [M:0;jenkins-hbase4:46447] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,46447,1699592001315 already deleted, retry=false 2023-11-10 04:56:20,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-10 04:56:20,335 INFO [M:0;jenkins-hbase4:46447] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46447,1699592001315; zookeeper connection closed. 2023-11-10 04:56:20,335 DEBUG [Listener at localhost/35765-EventThread] zookeeper.ZKWatcher(600): master:46447-0x101ce530b520000, quorum=127.0.0.1:51169, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-10 04:56:20,337 WARN [Listener at localhost/35765] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-11-10 04:56:20,340 INFO [Listener at localhost/35765] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-10 04:56:20,448 WARN [BP-1289017901-172.31.14.131-1699591997884 heartbeating to localhost/127.0.0.1:43735] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-11-10 04:56:20,449 WARN [BP-1289017901-172.31.14.131-1699591997884 heartbeating to localhost/127.0.0.1:43735] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1289017901-172.31.14.131-1699591997884 (Datanode Uuid ef526ac7-835b-4e32-8089-9ce0eb46f62e) service to localhost/127.0.0.1:43735 2023-11-10 04:56:20,452 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/cluster_48cecc52-0985-53ae-674b-7482adaea2d3/dfs/data/data1/current/BP-1289017901-172.31.14.131-1699591997884] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-10 04:56:20,452 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/481e751b-37e5-4b30-4cdb-e5f7b65d90ec/cluster_48cecc52-0985-53ae-674b-7482adaea2d3/dfs/data/data2/current/BP-1289017901-172.31.14.131-1699591997884] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-10 04:56:20,479 INFO [Listener at localhost/35765] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-10 04:56:20,613 INFO [Listener at localhost/35765] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-11-10 04:56:20,693 INFO [Listener at localhost/35765] hbase.HBaseTestingUtility(1293): Minicluster is down